Handling object references about scuttlebutt

[kumavis]

It Would Be Easy™ to add this functionality on top of scuttlebutt, but I'm curious if this already exists.

[dominictarr]

No, this doesn't already exist. Although,my replicated data store that predated
scuttlebutt - snob (which was a git-esque hash tree of diffs) I had a thing to maintain the structure of references, and detect cycles, etc.

(although, this required that each branch had and __id__ property, although you could just use weakmaps)

scuttlebutt was my 3rd attempt at a data replication system,
each simpler than the one before it.

I don't mean to be discouraging, but I think it's unwise to try to pave over distribution and make it feel like a single system. I think the bible has a section on distributed programming which says something like "God, grant me the serenity to accept the things I cannot change, The courage to change the things I can, And wisdom to know the difference"

I think distributed systems is one of these things you cannot change.
It's a much better base assumption that you are running across many systems.
The way things are going (massively multicore) sooner or later people will be using
eventually consistency techniques within a single (multicore) chip!

That is not to say that a scuttlebutt that replicates a tree structure -
which is what this boils down to, i think? is not an interesting and worthwhile problem.

What do you want to use it for?

[kumavis]

Ah hah yes, wise words.
I did have a feeling in the back of my mind that it would be an over simplification and paving over of the subtleties of the system.

As for why and what for, its really just an experiment. Mad science.
I'm new to thinking about things as distributed systems and was inspired by scuttlebutt.
I've been playing with voxel.js and multiplayer over webrtc, but started thinking of a system without a dedicated server.

Anyways I'll throw it together and see what its like to work with.

[dominictarr]

Okay, awesome! let me know if you have any questions!

Just on thing though:

//this pattern is better
duplex.pipe(sb.createStream()).pipe(duplex)

//than:
sb.handle(duplex)

Because you know that your thing is using streams by the correct api and not doing anything weird.
There is now quite a range of things that use this pattern, from dnode, rpc-stream to scuttlebutt and it's subclasses.

[kumavis]

+1 to that. Also frees your module consumer to pipe things in ways you didn't expect (like a throughput benchmarking transform).

[kumavis]

Heh, here it is: https://github.com/kumavis/synced-object-store

Unfortunately it doesn't work out of the box for things like handing a voxel.js game object, because of a reference to window somewhere in the tree.

While I can't imagine using this in a real project, 'twas a fun hack. Curious what your thoughts are.

[dominictarr]

@kumavis looks interesting. can you put a simple example of how to use it in the readme?