Comments (17)
OK, so a clarification. The messages are not coming concatenated. Rather data
is being emitted in for the form of a partial message (clipped stringified JSON) without a line ending and then the next data
event emitted is an entirely different update that ends with a line break and then that entire thing is parsed and errors.
from scuttlebutt.
Also, not sure if this is s Scuttlebutt problem or with CRDT.
from scuttlebutt.
could this have to do with max net packet sizes?
from scuttlebutt.
@juliangruber it must to be something like that but even so with TCP shouldn't the order should be preserved? I'm currently digging through the duplex models and the streams module itself to see if there's anyway for things to get out of order there.
from scuttlebutt.
Unless it's not an order problem at all but rather truncating and never sending the remainder.
from scuttlebutt.
I had this case once where
res.write('foobar');
didn't have the same effect as
res.write('foo');
res.write('bar');
but maybe that has been addressed in a newer node version?
from scuttlebutt.
If anywhere, this should be handled in https://github.com/dominictarr/stream-serializer/blob/master/index.js
stream-serializer.
Messages are appended together until a \n is found,
and then that item is json parsed and passed to the write stream that expected a regular object.
https://github.com/dominictarr/stream-serializer/blob/master/index.js#L32-L36
from scuttlebutt.
okay, I hate to say this but... "this should never happen"
what are you streaming this over?
is this over tcp or websockets?
this is inside fleet, right?
from scuttlebutt.
if something was writing out a rogue \n then this could happen. but I'm not sure what could be doing this...
from scuttlebutt.
I did a tcpdump and I can in fact see the data being split between two packets (sent by client). In the tcp stream I can see in between these packets the server happens to send a scuttlebutt digest to the client. It is the combination of the partial message from the client and the unrelated digest the server send to the client that get's JSON.parsed and errors.
So it seems that the data sent from the server is being emitted into the stream from the client?
from scuttlebutt.
@dominictarr this particular case is with seaport over tcp.
from scuttlebutt.
can you get on irc? this is obviously urgent (#stackvm on freenode)
from scuttlebutt.
oh, @juliangruber you shouldn't be writing to the stream directly
from scuttlebutt.
this issue was fixed in seaport. https://github.com/substack/seaport/pull/41
from scuttlebutt.
@dominictarr: that was in a http stream
from scuttlebutt.
@juliangruber with scuttlebutt?
from scuttlebutt.
nope, totally unrelated :D
from scuttlebutt.
Related Issues (20)
- Scuttlebutt#clone requires your scuttlebutts use constructor functions, don't take arguments or require modification on an instance HOT 5
- Pub/Sub broker on top of Scuttlebutt HOT 2
- Automatic network setup HOT 3
- Handling object references HOT 7
- Syncing trees HOT 1
- how to use it with peer to peer network settings(not client server)? HOT 34
- Does scuttlebutt/events omit events? HOT 2
- Documentation , dosnt include client side code for connecting to a tcp server HOT 2
- Documentation , dosnt include client side code for connecting to a tcp server
- Prevent tampering HOT 7
- Example data structure for rich text editing HOT 8
- Integrate scuttlebutt with Meteor HOT 1
- Will time stamp precision be an issue? HOT 4
- Building a singleton P2P network for syncing Scuttlebutts? HOT 2
- Sharding HOT 1
- Data sync issue HOT 3
- Encrypted message to all friends
- There is no `sync` event on SB or Model? HOT 3
- App Crashes When Open -MACOS
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scuttlebutt.