Comments (4)
Thanks for reporting, this definitely needs investigation.
from puppeteer-cluster.
On each launch() function called, 4 events have been adding to NodeJS's process
const listeners = [helper.addEventListener(process, 'exit', killChrome)];
if (handleSIGINT)
listeners.push(helper.addEventListener(process, 'SIGINT', () => {
killChrome();
process.exit(130);
}));
if (handleSIGTERM)
listeners.push(helper.addEventListener(process, 'SIGTERM', gracefullyCloseChrome));
if (handleSIGHUP)
listeners.push(helper.addEventListener(process, 'SIGHUP', gracefullyCloseChrome));
When puppeteer-cluster launch a lot of Worker, NodeJS will show MaxListenersExceededWarning
error
FYI: https://nodejs.org/api/events.html#events_eventemitter_defaultmaxlisteners
By default, a maximum of 10 listeners can be registered for any single event.
.. EventEmitters will print a warning if more than 10 listeners are added for a particular event. This is a useful default that helps finding memory leaks.
You can easily disable this warning by using process.setMaxListeners(n)
with n=20
in your case or more.
For advanced use, you can remove all event listeners exit
added by Puppeteer. And do some stuff like killChrome()
and gracefullyCloseChrome()
by yourself.
Note that 3 events SIGINT, SIGTERM, SIGHUP
can be disabled in launch()
options. Work in my case
const browser = await puppeteer.launch({
handleSIGINT = false,
handleSIGTERM = false,
handleSIGHUP = false,
});
from puppeteer-cluster.
Note that 3 events
SIGINT, SIGTERM, SIGHUP
can be disabled inlaunch()
options. Work in my caseconst browser = await puppeteer.launch({ handleSIGINT = false, handleSIGTERM = false, handleSIGHUP = false, });
FYI, most process managers use at least one of those signals to "ask" the process to terminate nicely. For example, systemd would send SIGTERM followed by SIGINT 90 seconds later (default behavior). Disabling them without explicitly call killChrome
or gracefullyCloseChrome
may spawn zombie processes.
raising max listeners to match worker count would be the safest workaround. As an added bonus, you could register additional listeners to do resource cleanup when the signal received.
from puppeteer-cluster.
For me removing '--single-process',
from args
fixed the issue
puppeteerOptions: {
headless: true,
args: [
// '--single-process',
],
}
from puppeteer-cluster.
Related Issues (20)
- Clear up Concurrency wording incorrect usage HOT 5
- Feature: Lifetimes
- How To Stop Worker To Become Idle automatically
- Expose stats via prometheus HOT 2
- Screen shot getting stuck forever
- Use same URL but diffetent logic on each browser HOT 1
- Concurrency launch: CONCURRENCY_BROWSER definition slightly misleading HOT 2
- Error detection super slow with new Puppeteer versions HOT 1
- Support to new puppetter versions HOT 1
- share the dockerfile I'm using
- Suggestion: Allow pool of already instantiated browser workers
- how to open the progress view and monitoring statistics? HOT 1
- cluster concurrent seems not work HOT 1
- Regarding resource usage HOT 1
- Worker Error getting browser page HOT 1
- How to set args like .launch({ args: [] }) ? HOT 3
- Has anyone managed to use separate data for each browser?
- browser crushing due to "open /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq: No such file or directory (2)"
- puppeteer-v22.0.0 breaking changes HOT 2
- concurrent limit?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from puppeteer-cluster.