jpaulm / jsfbp Goto Github PK

@ComFreek - I like the way you have split up index.js, and I have updated the Readme to reflect that getElementWithSmallestBacklog is in utils, but I am wondering why InitConn is also in utils, as it is not exposed to components (although it is used by initialize)... Just wondering! Thx.

Line 62 in Network.js incorrectly identifies connections to empty IIPs as being Process connections

I recently moved to Win10, so I thought I should do some retesting of JSFBP. I have been struggling with this nonsense for over a week, and do not seem to be any closer.

I believe that I need Python 2.7, and Visual Studio Tools 17.0. I have (probably) reinstalled gyp hundred times. I usually get a message about Retargetting to Tools V141, but this seems to have gone away with Visual Studio Tools 17.0, but that's about it!

Could someone try installing fibers, and let me know how you did it...?!

TIA,

Paul

PS @ComFreek , npm used to be a stable ecosystem, but I can't say that is true any more! It shouldn't be this difficult!

Some notes and findings (a.k.a. a TODO list)

collate.js missing logic to insert open and close brackets

Is it possible to see which input ports have IPs ready to be received? Or to receive on multiple ports at the same time, and take whichever receives an IP first? (like select in POSIX sockets)

Sorry I fat fingered the Submit button, editing the description now. I really like the simplicity of this project compared to something like noflo. But noflo has proper HTTP in addition to web socket support, which is I think is needed to make a truly decent web chat demo that I will to work on (as time permits). I plan to push some more fixes, opening this for general discussion.

• [variableName] = [value] creates a global variable if variableName hasn't been declared before.
  This pollutes the global namespace and should therefore be avoided. An example can be found in fbp.js#19.
• Use module.exports instead of exports
  Sources:
  • http://justbuildsomething.com/node-js-best-practices/#9
  • http://stackoverflow.com/a/7142924/603003
• "use strict" should be used (as in this file)
• Maybe the code base would benefit from a JS dialect with a type system (e.g. TypeScript).

I would submit a PR for the first three things if you're okay with that.

@jpaulm @tlrobinson @ComFreek

What do you guys think about how local state should be handled? It seems like the current JSFBP only allows "pure" processes. If you save the state outside of the scope of the module.exports function it becomes a singleton process.

I'm thinking it could be something like this, in pseudo-JS:

module.exports = function count () {
  var proc = getProcess();
  var inport = openInputPort('in');
  var count = proc.getState('count');

  while (true) {
    var ip = receive();
    if (ip === null) {
      break;
    }

    count++;
    proc.putState('count', count);
    send(newIP(count));
  }
};

Any idea or previous syntax that I'm not aware of for handling local states?

Travis seems to be broken as of 44bd142. As far as I can see the only change was to the eslint version!

Any ideas, @ComFreek?

Best regards, and season's greetings!

Trying to get started on Win 7 by following the README does not work.

What gives?

@bjouhier Probably dumb question. As I understand it, each 'run' call will run until the code returns, or until a 'yield' occurs - my question is how do I tell the difference between these two cases? Of course, I may not have understood correctly. Thanks!

The readme of node-fibers suggests to create an abstraction layer:

The examples below describe basic use of Fiber , but note that it is not recommended to use Fiber without an abstraction in between your code and fibers. See "FUTURES" below for additional information.

_{— source: https://github.com/laverdet/node-fibers#sleep}

Fortunately, Fibers is (yet) only used in index.js and in core files (in the code base of my PR). "User land code" should ideally never require Fiber calls.

Test network testsubstreamsensitivesplitting.js should stop with a deadlock when the connection capacities are set to a small number - strangely enough, it doesn't, although the same network in JavaFBP and C#FBP (preemptive multithreading) does.

Challenge to the reader: why doesn't it deadlock? TIA

We use basic arrays in a number of places where associative arrays are more appropriate.

I'm working on changing that.

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

Since flowbased/fbp is a thing (and thus, the parsing is a done deal),it would be nice if JSFBP supported
.fbp files.

Regarding composite components

Is it intended that composite components are simply components that contain Networks? Is this the right approach?

For example Mocha or any other JS unit testing framework.

I'd like to see a roadmap/goals/features list to have the project on track for contributions.

@bjouhier Sorry to keep bothering you, but I'm a bit confused about running a callback under a fibre - not even sure if that's the right terminology!

I had my test case with the asynchronous reader working (I thought), but when I tried a more complex case (fbptest2), I found that the callback wasn't picking up the original fibre. I now seem to have gotten it working - with some added logic in reader.js, and some more functions in fbp.js, but I just wanted to ask what controls the fibre's lifetime? I seem to have been able to reconnect to my original fibre from the callback, using Fiber.current = xxx, but I don't know how that works under the covers!

Hope this makes sense! I'd like to be a bit more comfortable with what I've done!

Best regards, and thanks for all your help!

Paul

Fibers are neat, but the majority of node.js developers don't and won't use them, and they don't work in browsers at all. It would be nice if we could provide a "classical FBP" system that works using standard JavaScript APIs.

Actually, after reading http://www.jpaulmorrison.com/fbp/noflo.html I'm pretty confused about how JSFBP is considered more "classical" than NoFlo. That page suggests the key difference is that NoFlow is single-threaded (because JavaScript is single-threaded) and thus not suitable for CPU-intensive programs, but then admits JSFBP is also single-threaded. So is the main difference actually the synchronous vs asynchronous style of programming?

Selective reading of ports and backpressure can both be accomplished in an asynchronous style. For example, Node-style Streams have pause and resume methods for handling backpressure. The port could buffer up to a fixed number of IPs then call pause on the stream when the buffer fills up. The port could have a read method that takes a callback (or returns a promise) which gets called when the next IP is available, and calls resume on the stream if the buffer is no longer.

To take advantage of multiple cores you could use something like WebWorkers, though I think a generic system based on a standard IPC mechanism (e.x. ZeroMQ) and serialization protocol (e.x. Protocol Buffers) would be even cooler, allowing you to write components in any language, or even distribute processes across machines.

But anyway, there are several options for sync and async styles in JavaScript:

Asynchronous style:

1. EventEmitters or Streams (see above)
2. raw callbacks: probably Node-style callback(err, result) callbacks. This is my least favorite option, but is still the most popular style in Node.js. I probably wouldn't bother with this.
3. promises: promises are gaining a lot of traction in the JavaScript world lately, but they're still a very asynchronous style

Synchronous style:

1. generators (ES6, can be transpiled to ES5): not available in browsers except via transpiling
2. await (ES7, can be transpiled to ES5): lets you write synchronous style code that awaits on promises
3. some other synchronous-style transpiler: e.x. streamline.js

Also this may be useful relevant reading: https://github.com/kriskowal/gtor

EDIT: task.js is another possibility: http://taskjs.org/

I've created a branch (bugfix/support-iip-and-output-to-inport) which shows some behaviour that I'm not sure about.

I have the following network:

1 -> IN RECVR(MockReceiver)
50 -> INTVL DELAY(jsfbp/delay)
2 -> IN DELAY
DELAY OUT -> IN RECVR

I expected to see the result [1, 2].

Instead, the test times out. When you switch on tracing, it looks like RECVR constantly reads 1 from its IN port.

Is this correct behaviour? Should a port connected to an IIP constantly be able to read that IIP from that port? I would have expected that port to return null once the IIP has been received.

If you look at https://github.com/jpaulm/jsfbp/blob/master/examples/update_c.js , you will see 3 occurrences of the reader.js component. The 2nd parameter of the defProc calls seems to be there just to produce meaningful list items in the trace log, if one is requested. OTOH it means thinking up what is really a 3rd name for the process. I would suggest that the name simply be generated using func plus a number, thus removing the requirement for a 2nd parameter on defProc. Comments, anyone?

Hello Paul,

I have just created a package.json for this project. It includes the fibers dependency, so that the installation will become as simple as git clone ... and npm install in the cloned directory. I plan to submit a PR.

Every package.json should include a license field. What is the license for the code in this repo?

Need to reinstate send suspend logic - this seems to be mostly in PR #74

What are your thoughts about adding this to the Node Package Manager system, so that folks can use it that way.

I know that npm supports adding dependencies directly from github, but a distribution version of this would be great!

I'm more than happy to do the legwork here.

@tlrobinson @ComFreek

I've been trying to tackle back pressure in my fork, and I must say I'm hitting a dead-end and it'd be nice to hear your thoughts on this.

Take a look at this gist. Replace the content of recvr.js on @ComFreek 's fork and run fbptest01.js. You should see something similar to this. The timing doesn't matter, but (correct me if I'm wrong please @jpaulm ) the copier should stop after the 5th copy is sent, because of a capacity of 5 set on the connetion between copier and recvr.

I know that @ComFreek might not have gotten to that point in his fork. Me neither. But the issue is that it seems to be rather difficult in JS (at least with ES5 as that was my thesis).

With generators, the control flow is inverted here so there doesn't seem to be an obvious way to have the callee of yield to suspend, or continue to suspend, the caller (in this case, the FBP process). In this particular case, yield is returned a bluebird promise but Promise.coroutine() would continue the generator as soon as receive settles, which it must do. I haven't put too much thought into this problem in a streams-based solution, but I suspect that it's a similar situation.

Any idea on this problem?

The latest NPM version is 2.1.0 (published 3 years ago), the version specified in package.json in latest commits is 1.x.y (for some values of x, y I forgot). That version mismatch is particularly confusing.

What do you think about bumping the version in package.json to 3.0.0 and releasing a new NPM package version?

This issue provides visibility into Renovate updates and their statuses. Learn more

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency mocha to v9
• chore(deps): update dependency ws to v8

• Check this box to trigger a request for Renovate to run again on this repository

File: components/reader.js
Use the err parameter passed to the callback:

fs.readFile(path, options, function(err, data) {

The collate test is currently failing:

$ npm test                                                                                                                                                           (master) 

> [email protected] test /Users/danrumney/Projects/jsfbp
> mocha --ui mocha-fibers --recursive -R spec --require test/test_helper.js



  collate
Start time: 5/25/2016, 10:55:24 AM
Elapsed time in millisecs: 8
    1) should collate based on a single field

  copier
Start time: 5/25/2016, 10:55:24 AM
Elapsed time in millisecs: 0
    ✓ should copy multiple IPs

  delay
Start time: 5/25/2016, 10:55:24 AM
Elapsed time in millisecs: 1003
    ✓ should exactly delay a single IP (1005ms)
Start time: 5/25/2016, 10:55:25 AM
Elapsed time in millisecs: 3040
    ✓ should exactly delay multiple IPs (3040ms)

  randdelay
Start time: 5/25/2016, 10:55:29 AM
Elapsed time in millisecs: 106
    ✓ should randomly delay a single IP (108ms)
Start time: 5/25/2016, 10:55:29 AM
Elapsed time in millisecs: 3715
    ✓ should randomly delay multiple IPs (3717ms)

  FiberRuntime
Start time: 5/25/2016, 10:55:32 AM
Elapsed time in millisecs: 0
    ✓ should be able to run a simple sender-->receiver setup

  utils
    ✓ should create an enum and provide a lookup function
    ✓ should not be writable


  8 passing (8s)
  1 failing

  1) collate should collate based on a single field:

      Uncaught AssertionError: expected [ Array(18) ] to deeply equal [ Array(10) ]
      + expected - actual

       [
      -  [null]
         "1,m1"
         "1,d11"
         "1,d12"
      -  [null]
      -  [null]
         "2,m2"
         "2,d21"
      -  [null]
      -  [null]
         "3,m3"
         "3,d31"
         "3,d32"
         "3,d33"
      -  [null]
      -  [null]
         "4,d41"
      -  [null]
       ]

      at Assertion.assertEqual (node_modules/chai/lib/chai/core/assertions.js:485:19)
      at Assertion.ctx.(anonymous function) [as equal] (node_modules/chai/lib/chai/utils/addMethod.js:41:25)
      at Context.<anonymous> (test/components/collate.js:24:28)
      at node_modules/mocha-fibers/lib/mocha-fibers.js:24:14



npm ERR! Test failed.  See above for more details.
[danrumney:~/Projects/jsfbp]$                         

It looks like the null IPs are being propagated instead of dropped.

setTimeout is an asynchronous operation (the passed in callback gets called after the delay), so this doesn't actually delay anything: https://github.com/jpaulm/jsfbp/blob/master/script/randdelay.js#L24-L28

I'm not sure what the correct implementation should be. I tried something like this https://github.com/laverdet/node-fibers#sleep which seemed to correctly delay, but then the process never exits.

I'd suggest setting up Renovate to prevent outdating of dependencies again.

See https://github.com/apps/renovate.

It's good form to avoid explicit dependencies on JSFBP objects.

Right now, users need to depend on the IP object in order to create brackets.

Let's expose those types on the Process object so that they are accessible to component creators.