pgriess / node-webworker Goto Github PK

I don't know why all the child workers exit with a code 0 and signal null in 2 minutes.

And I tried to restart worker by binding a function to worker.onexit, it works, but still don't know why..

Since this socket is only used to get a connection from a single client, we don't need to keep it open after we get the first 'connection' event. Leaving it open consumes an FD and other resources for the WSS instance.

Error on require('webworker')

Error: No such module
    at Object.<anonymous> (/node_modules/webworker/lib/webworker.js:35:26)

Offending line var netBinding = process.binding('net'); yields same error.

As of nodejs/node-v0.x-archive@75db199 Script is renamed to NodeScript in process.bindings('eval').

You should be able to fix this for both Node 0.4.x and 0.5.0-pre by changing var script = process.binding('evals'); to var script = require('vm'); in lib/webworker-child.js.

I'd send you a pull request but I don't feel like installing 0.4.x to test on the stable branch. The modification I mentioned works on 0.5.0-pre though.

Running node 0.4.9, terminate() calls fail at the first assertion. If I log the pid to console when it is re-set after spawn, it shows up fine. If I store it as an attribute on the worker object and try to reference it there, it always comes back undefined. Here's the code that allows me to reproduce the != fail on the first line of the terminate method

var Worker = require('webworker').Worker
var worker = new Worker(__dirname+'/worker.js');
worker.terminate();

Any thoughts?

Hello,

Just following the README example I got:

TypeError: object is not a function

The fix I found for this is:

var Worker = requre('webworker').Worker

Not sure why this is happening, the ./lib file correctly exports Worker as it's module.exports.

Thanks

Just the line where it is explained how to install this via npm

net.js:1016
throw new Error('Non-socket exists at ' + path);
^
Error: Non-socket exists at /tmp/node-webworker-88948
at net.js:1016:17

There are several steps that the Worker instance has to go through before the child process is fully established: waiting for the 'listening' event; waiting for the 'connection' event, etc.

Make sure that we properly clean up state no matter where we are in the lifetime of the child process.

I'm wondering if OSX is supported at this point?

I can't get your example from the mailing list to work (connections seem to be accepted, but not responded to).

When running the test suite you ship, I can only run test-simple. test-error and test-fd just seem to stall forever.

If you need help with osx I can probably try looking into this.

/home/splincode/Develop/test-selenuim/node_modules/webworker/lib/webworker.js:35
var netBinding = process.binding('net');
^

Error: No such module: net

Node v 7.10

I'd like to ask if you could maybe provide a way to allow starting web workers with debug ports, e.g.:

diff --git a/lib/webworker.js b/lib/webworker.js
index 95f7489..60d5701 100644
--- a/lib/webworker.js
+++ b/lib/webworker.js
@@ -47,7 +47,7 @@ var numWorkersCreated = 0;
 //
 // Each worker communicates with the master over a UNIX domain socket rooted
 // in SOCK_DIR_PATH.
-var Worker = function(src) {
+var Worker = function(src, debugPort) {
     var self = this;

     // The timeout ID for killing off this worker if it is unresponsive to a
@@ -109,13 +109,15 @@ var Worker = function(src) {
     // and away we go.
     var start = function() {
         srv.addListener('listening', function() {
+            const dbg = debugPort ? ["--debug=" + debugPort] : []
             cp = child_process.spawn(
                 process.argv[0],
+                dbg.concat(
                 [
                     path.join(__dirname, 'webworker-child.js'),
                     sockPath,
                     'file://' + src
-                ],
+                ]),
                 undefined,
                 [0, 1, 2]
             );

This is currently hard-coded to process.argv[0](really we should use process.execPath anyway). Instead, we should allow the caller to override this using the new 'opts' argument to the Worker() constructor.

Witness:

Error: ENOENT, No such file or directory '/child.js'
    at Object.openSync (fs:150:18)
    at Object.readFileSync (fs:94:15)
    at Object.<anonymous> (/opt/local/lib/node/webworker/webworker-child.js:138:12)
    at Module._compile (node.js:472:25)
    at Module._loadScriptSync (node.js:482:10)
    at Module.loadSync (node.js:352:12)
    at Object.runMain (node.js:535:24)
    at node.js:760:10

I create workers with this bit of code:

for (var i = 0; i < MAX_WORKERS; i++){
  arr[i] = new Worker(CURRENT_DIR + '/lib/workers/worker.js');

  arr[i].onmessage = function(msg, cb) { ... }
  }

  arr[i].onexit = function(code, syssignal){
    console.log("\033[1;32m#### WORKER TERMINATED, signal %s #####\033[0m", syssignal);
  };

}

My intention is to use long-lived workers and rotate through them (I'm using them for file upload handling). Is this the right approach?

Second, these workers terminate with signal null when server is under heavy load (not file uploads, but just GET /). Why would they do that and how can I figure out why they crash?

I ran into a situation where I had 9999 such files left in /tmp and no more webworkers would be able to start. There should be a periodic garbage collection of those ?

Installing with node v0.4.8, npm 1.0.15, centOS 5.5-64. Getting following error:

npm ERR! tar "-mvxpf" "-" "-o" "-C" "/tmp/npm-1310337053203/1310337053203-0.9732671466190368/contents/___package.npm" tar: Ignoring unknown extended header keyword LIBARCHIVE.creationtime' npm ERR! tar "-mvxpf" "-" "-o" "-C" "/tmp/npm-1310337053203/1310337053203-0.9732671466190368/contents/___package.npm" tar: Ignoring unknown extended header keywordSCHILY.dev'
npm ERR! tar "-mvxpf" "-" "-o" "-C" "/tmp/npm-1310337053203/1310337053203-0.9732671466190368/contents/___package.npm" tar: Ignoring unknown extended header keyword SCHILY.ino' npm ERR! tar "-mvxpf" "-" "-o" "-C" "/tmp/npm-1310337053203/1310337053203-0.9732671466190368/contents/___package.npm" tar: Ignoring unknown extended header keywordSCHILY.nlink'

and then it repeats until:

npm ERR! tar "-mvxpf" "-" "-o" "-C" "/tmp/npm-1310337053203/1310337053203-0.9732671466190368/contents/___package.npm" tar: Error exit delayed from previous errors
npm ERR! tar "-mvxpf" "-" "-o" "-C" "/tmp/npm-1310337053203/1310337053203-0.9732671466190368/contents/___package.npm"
npm ERR! Failed unpacking /tmp/npm-1310337053203/1310337053203-0.9732671466190368/tmp.tgz
npm ERR! couldn't unpack /tmp/npm-1310337053203/1310337053203-0.9732671466190368/tmp.tgz to /tmp/npm-1310337053203/1310337053203-0.9732671466190368/contents
npm ERR! Error: Failed tar "-mvxpf" "-" "-o" "-C" "/tmp/npm-1310337053203/1310337053203-0.9732671466190368/contents/___package.npm"
npm ERR! exited with 2
npm ERR! at ChildProcess. (/usr/local/lib/node_modules/npm/lib/utils/exec.js:85:8)
npm ERR! at ChildProcess.emit (events.js:67:17)
npm ERR! at ChildProcess.onexit (child_process.js:192:12)
npm ERR! Report this entire log at:
npm ERR! http://github.com/isaacs/npm/issues
npm ERR! or email it to:
npm ERR! [email protected]
npm ERR!
npm ERR! System Linux 2.6.18-194.11.3.el5
npm ERR! command "node" "/usr/local/bin/npm" "install" "websocket-client"
npm ERR! cwd /home/admin
npm ERR! node -v v0.4.8
npm ERR! npm -v 1.0.15
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /home/admin/npm-debug.log
npm not ok

Same thing happens when doing npm install websocket-client. Possibly related?

Michael

Running the example from the README (node v0.4.7), I get:

scott@vscott:~/devel/js/node/webworker-tests$ ./master.js 

node.js:134
        throw e; // process.nextTick error, or 'error' event on first tick
        ^
TypeError: object is not a function
    at Object.CALL_NON_FUNCTION_AS_CONSTRUCTOR (native)
    at Object.<anonymous> (/home/scott/devel/js/node/webworker-tests/master.js:6:9)
    at Module._compile (module.js:404:26)
    at Object..js (module.js:410:10)
    at Module.load (module.js:336:31)
    at Function._load (module.js:297:12)
    at Array.<anonymous> (module.js:423:10)
    at EventEmitter._tickCallback (node.js:126:26)

Module was installed globally using npm 1.0.6.

Additionally, the readme says websocket-client 0.9.3 or later is required, package.json (on github) says 0.9.3 - 0.9.9999, and my package.json (from npm) does not have a dependencies entry.

I installed the latest version of websocket-client (1.0.0) globally using npm but it had no effect on the above error.

A npm package would be very nice for this : ).

Hello

Are there any plans to get node-webworker running on node 0.8.x? I'm currently getting the following error on node 0.8.0 and webworker 0.8.4. Is the cluster module a correct replacement for webworkers? Thanks!

/Users/rgermano/node_modules/webworker/lib/webworker.js:35
var netBinding = process.binding('net');
^
Error: No such module
at Object. (/Users/rgermano/node_modules/webworker/lib/webworker.js:35:26)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:362:17)
at require (module.js:378:17)
at Object. (/Users/rgermano/iG/workspaces/cms/git/iGCMS_EdicaoHomes/lib/publicacao/controle.js:10:14)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)

How can one redirect stdout of child worker to somewhere (to the master`s stdout)? As it seems in the code there is no such a chance and the process/pid object is hidden inside of the Worker. May be process (cp) variable should be exposed with some getter to allow requested feature?

Rather than relying on disk IO, it'd be convenient to pass a function as follows to construct the WebWorker:

const Worker = require('webworker');
const worker = new Worker(function (msg) {
  return `message was: "${msg}" (${msg.length} bytes)`;
});

Hi,

I have following simple program. I am looking to parallelize this program using web-worker.

function process(delay){
var start = new Date();
var len = 300000*delay;
for(var i=0;i<len;i++){};
return (new Date()-start);
}

var start = new Date();
process(500);
process(600);
console.log("Time taken :"+(new Date()-start));

When my child worker have some syntax error, or runtime error which outside onmessage, onerror callback, the parent worker does not show any message, just wait there.

Today I just use process.env.NODE_DEBUG = 0x8 to open webworker's debug mode. then I can see wwutil's debug infomation, it does have some debug information when the worker wasn't started correctly, is there any function I can call or event I can listen to and print out those errors(instead just open webworker's debug mode as I am doing now.)?

var Worker;
while (!Worker) {
console.log(Worker);
Worker = require('webworker').Worker;
}
console.dir(Worker);

outputs:

undefined
undefined
[Function]

instead of:

undefined
[Function]

Took me a day to realize it. Any idea what might cause this?

In the README.md, Line 6 there is:

See the design document here.

With a broken link to: http://blog.std.in/2010/07/08/nodejs-webworker-design/

I can't post a pull request because I don't know what the correct link should be.

It seems to be broken since node 0.8

This event should probably include the exit code and signal.

I'm trying to use the webworker module with node 0.6.7 but I can't get it to work.

I've made the edits suggested earlier to fixe 0.6 compatibility, but all the 3 tests in the test folder fail because the assertions are not passed.

Can anyone let me know that version of node DOES work with this? Is anyone still actively developing this? A lot of simple open issues that haven't been fixed it seems.

since the coffeescript wraps every file it compiles with a lambda block:
(function() {
...
})
all handlers created in the global scope in the coffeescript file (onmessage,onerror) are actually not global but wrapped in that same lambda block, and therefore not visible to the webworker-child.js setup script... hence workerCtx.onmessage is undefined, and so is onerror, and so the worker does not receive messages from the master...

coffeescript is gaining widespread use, and their decision to wrap every script in a lambda block to protect the global namespace is a valid one. hence I think that, in node-webworker, onmessage/onerror handlers should not be declared as they are currently as global vars, but instead should be passed in to some sort of global init() function (that would be attached to workerCtx when the worker is being setup inside webworker-child.js). that way it wouldnt matter where they're declared...

Right now it's difficult/impossible for worker scripts to use command line arguments in any meaningful way. The contents of the process.argv[] array contain webworker-child.js and . It'd be nice to strip this stuff out so that the worker scripts could be passed arguments on the command line more easily.

This would be useful!

Right now, we have code in webworker-child.js that explicitly adds a hard-coded list of globals into the child context. When new globals get added with additional node releases, this code breaks (e.g. the new console global is not available).

We should programatically iterate over all globals and copy them.