jpaulm / jsfbp Goto Github PK
View Code? Open in Web Editor NEWFBP implementation written using JavaScript and node-fibers
License: MIT License
FBP implementation written using JavaScript and node-fibers
License: MIT License
@ComFreek - I like the way you have split up index.js, and I have updated the Readme to reflect that getElementWithSmallestBacklog is in utils, but I am wondering why InitConn is also in utils, as it is not exposed to components (although it is used by initialize)... Just wondering! Thx.
Line 62 in Network.js incorrectly identifies connections to empty IIPs as being Process connections
I recently moved to Win10, so I thought I should do some retesting of JSFBP. I have been struggling with this nonsense for over a week, and do not seem to be any closer.
I believe that I need Python 2.7, and Visual Studio Tools 17.0. I have (probably) reinstalled gyp hundred times. I usually get a message about Retargetting to Tools V141, but this seems to have gone away with Visual Studio Tools 17.0, but that's about it!
Could someone try installing fibers, and let me know how you did it...?!
TIA,
Paul
PS @ComFreek , npm used to be a stable ecosystem, but I can't say that is true any more! It shouldn't be this difficult!
collate.js
missing logic to insert open and close brackets
Is it possible to see which input ports have IPs ready to be received? Or to receive on multiple ports at the same time, and take whichever receives an IP first? (like select
in POSIX sockets)
Sorry I fat fingered the Submit button, editing the description now. I really like the simplicity of this project compared to something like noflo. But noflo has proper HTTP in addition to web socket support, which is I think is needed to make a truly decent web chat demo that I will to work on (as time permits). I plan to push some more fixes, opening this for general discussion.
[variableName] = [value]
creates a global variable if variableName
hasn't been declared before.module.exports
instead of exports
I would submit a PR for the first three things if you're okay with that.
What do you guys think about how local state should be handled? It seems like the current JSFBP only allows "pure" processes. If you save the state outside of the scope of the module.exports
function it becomes a singleton process.
I'm thinking it could be something like this, in pseudo-JS:
module.exports = function count () {
var proc = getProcess();
var inport = openInputPort('in');
var count = proc.getState('count');
while (true) {
var ip = receive();
if (ip === null) {
break;
}
count++;
proc.putState('count', count);
send(newIP(count));
}
};
Any idea or previous syntax that I'm not aware of for handling local states?
Trying to get started on Win 7 by following the README does not work.
What gives?
@bjouhier Probably dumb question. As I understand it, each 'run' call will run until the code returns, or until a 'yield' occurs - my question is how do I tell the difference between these two cases? Of course, I may not have understood correctly. Thanks!
The readme of node-fibers suggests to create an abstraction layer:
The examples below describe basic use of Fiber , but note that it is not recommended to use Fiber without an abstraction in between your code and fibers. See "FUTURES" below for additional information.
— source: https://github.com/laverdet/node-fibers#sleep
Fortunately, Fibers is (yet) only used in index.js and in core files (in the code base of my PR). "User land code" should ideally never require Fiber calls.
Test network testsubstreamsensitivesplitting.js
should stop with a deadlock when the connection capacities are set to a small number - strangely enough, it doesn't, although the same network in JavaFBP and C#FBP (preemptive multithreading) does.
Challenge to the reader: why doesn't it deadlock? TIA
We use basic arrays in a number of places where associative arrays are more appropriate.
I'm working on changing that.
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.
Since flowbased/fbp is a thing (and thus, the parsing is a done deal),it would be nice if JSFBP supported
.fbp files.
Regarding composite components
Is it intended that composite components are simply components that contain Networks? Is this the right approach?
For example Mocha or any other JS unit testing framework.
I'd like to see a roadmap/goals/features list to have the project on track for contributions.
@bjouhier Sorry to keep bothering you, but I'm a bit confused about running a callback under a fibre - not even sure if that's the right terminology!
I had my test case with the asynchronous reader working (I thought), but when I tried a more complex case (fbptest2), I found that the callback wasn't picking up the original fibre. I now seem to have gotten it working - with some added logic in reader.js, and some more functions in fbp.js, but I just wanted to ask what controls the fibre's lifetime? I seem to have been able to reconnect to my original fibre from the callback, using Fiber.current = xxx, but I don't know how that works under the covers!
Hope this makes sense! I'd like to be a bit more comfortable with what I've done!
Best regards, and thanks for all your help!
Paul
Fibers are neat, but the majority of node.js developers don't and won't use them, and they don't work in browsers at all. It would be nice if we could provide a "classical FBP" system that works using standard JavaScript APIs.
Actually, after reading http://www.jpaulmorrison.com/fbp/noflo.html I'm pretty confused about how JSFBP is considered more "classical" than NoFlo. That page suggests the key difference is that NoFlow is single-threaded (because JavaScript is single-threaded) and thus not suitable for CPU-intensive programs, but then admits JSFBP is also single-threaded. So is the main difference actually the synchronous vs asynchronous style of programming?
Selective reading of ports and backpressure can both be accomplished in an asynchronous style. For example, Node-style Streams have pause
and resume
methods for handling backpressure. The port could buffer up to a fixed number of IPs then call pause
on the stream when the buffer fills up. The port could have a read
method that takes a callback (or returns a promise) which gets called when the next IP is available, and calls resume
on the stream if the buffer is no longer.
To take advantage of multiple cores you could use something like WebWorkers, though I think a generic system based on a standard IPC mechanism (e.x. ZeroMQ) and serialization protocol (e.x. Protocol Buffers) would be even cooler, allowing you to write components in any language, or even distribute processes across machines.
But anyway, there are several options for sync and async styles in JavaScript:
Asynchronous style:
callback(err, result)
callbacks. This is my least favorite option, but is still the most popular style in Node.js. I probably wouldn't bother with this.Synchronous style:
await
s on promisesAlso this may be useful relevant reading: https://github.com/kriskowal/gtor
EDIT: task.js is another possibility: http://taskjs.org/
I've created a branch (bugfix/support-iip-and-output-to-inport) which shows some behaviour that I'm not sure about.
I have the following network:
1 -> IN RECVR(MockReceiver)
50 -> INTVL DELAY(jsfbp/delay)
2 -> IN DELAY
DELAY OUT -> IN RECVR
I expected to see the result [1, 2]
.
Instead, the test times out. When you switch on tracing, it looks like RECVR
constantly reads 1
from its IN
port.
Is this correct behaviour? Should a port connected to an IIP constantly be able to read that IIP from that port? I would have expected that port to return null
once the IIP has been received.
If you look at https://github.com/jpaulm/jsfbp/blob/master/examples/update_c.js , you will see 3 occurrences of the reader.js
component. The 2nd parameter of the defProc
calls seems to be there just to produce meaningful list items in the trace log, if one is requested. OTOH it means thinking up what is really a 3rd name for the process. I would suggest that the name simply be generated using func
plus a number, thus removing the requirement for a 2nd parameter on defProc
. Comments, anyone?
Hello Paul,
I have just created a package.json for this project. It includes the fibers
dependency, so that the installation will become as simple as git clone ...
and npm install
in the cloned directory. I plan to submit a PR.
Every package.json should include a license field. What is the license for the code in this repo?
Need to reinstate send suspend logic - this seems to be mostly in PR #74
What are your thoughts about adding this to the Node Package Manager system, so that folks can use it that way.
I know that npm supports adding dependencies directly from github, but a distribution version of this would be great!
I'm more than happy to do the legwork here.
I've been trying to tackle back pressure in my fork, and I must say I'm hitting a dead-end and it'd be nice to hear your thoughts on this.
Take a look at this gist. Replace the content of recvr.js
on @ComFreek 's fork and run fbptest01.js
. You should see something similar to this. The timing doesn't matter, but (correct me if I'm wrong please @jpaulm ) the copier should stop after the 5th copy is sent, because of a capacity of 5 set on the connetion between copier and recvr.
I know that @ComFreek might not have gotten to that point in his fork. Me neither. But the issue is that it seems to be rather difficult in JS (at least with ES5 as that was my thesis).
With generators, the control flow is inverted here so there doesn't seem to be an obvious way to have the callee of yield to suspend, or continue to suspend, the caller (in this case, the FBP process). In this particular case, yield
is returned a bluebird promise but Promise.coroutine()
would continue the generator as soon as receive
settles, which it must do. I haven't put too much thought into this problem in a streams-based solution, but I suspect that it's a similar situation.
Any idea on this problem?
The latest NPM version is 2.1.0
(published 3 years ago), the version specified in package.json
in latest commits is 1.x.y
(for some values of x, y I forgot). That version mismatch is particularly confusing.
What do you think about bumping the version in package.json
to 3.0.0
and releasing a new NPM package version?
This issue provides visibility into Renovate updates and their statuses. Learn more
These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
File: components/reader.js
Use the err
parameter passed to the callback:
fs.readFile(path, options, function(err, data) {
The collate test is currently failing:
$ npm test (master)
> [email protected] test /Users/danrumney/Projects/jsfbp
> mocha --ui mocha-fibers --recursive -R spec --require test/test_helper.js
collate
Start time: 5/25/2016, 10:55:24 AM
Elapsed time in millisecs: 8
1) should collate based on a single field
copier
Start time: 5/25/2016, 10:55:24 AM
Elapsed time in millisecs: 0
✓ should copy multiple IPs
delay
Start time: 5/25/2016, 10:55:24 AM
Elapsed time in millisecs: 1003
✓ should exactly delay a single IP (1005ms)
Start time: 5/25/2016, 10:55:25 AM
Elapsed time in millisecs: 3040
✓ should exactly delay multiple IPs (3040ms)
randdelay
Start time: 5/25/2016, 10:55:29 AM
Elapsed time in millisecs: 106
✓ should randomly delay a single IP (108ms)
Start time: 5/25/2016, 10:55:29 AM
Elapsed time in millisecs: 3715
✓ should randomly delay multiple IPs (3717ms)
FiberRuntime
Start time: 5/25/2016, 10:55:32 AM
Elapsed time in millisecs: 0
✓ should be able to run a simple sender-->receiver setup
utils
✓ should create an enum and provide a lookup function
✓ should not be writable
8 passing (8s)
1 failing
1) collate should collate based on a single field:
Uncaught AssertionError: expected [ Array(18) ] to deeply equal [ Array(10) ]
+ expected - actual
[
- [null]
"1,m1"
"1,d11"
"1,d12"
- [null]
- [null]
"2,m2"
"2,d21"
- [null]
- [null]
"3,m3"
"3,d31"
"3,d32"
"3,d33"
- [null]
- [null]
"4,d41"
- [null]
]
at Assertion.assertEqual (node_modules/chai/lib/chai/core/assertions.js:485:19)
at Assertion.ctx.(anonymous function) [as equal] (node_modules/chai/lib/chai/utils/addMethod.js:41:25)
at Context.<anonymous> (test/components/collate.js:24:28)
at node_modules/mocha-fibers/lib/mocha-fibers.js:24:14
npm ERR! Test failed. See above for more details.
[danrumney:~/Projects/jsfbp]$
It looks like the null
IPs are being propagated instead of dropped.
setTimeout is an asynchronous operation (the passed in callback gets called after the delay), so this doesn't actually delay anything: https://github.com/jpaulm/jsfbp/blob/master/script/randdelay.js#L24-L28
I'm not sure what the correct implementation should be. I tried something like this https://github.com/laverdet/node-fibers#sleep which seemed to correctly delay, but then the process never exits.
I'd suggest setting up Renovate to prevent outdating of dependencies again.
It's good form to avoid explicit dependencies on JSFBP objects.
Right now, users need to depend on the IP object in order to create brackets.
Let's expose those types on the Process object so that they are accessible to component creators.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.