linearregression / scuttlebot

a peer-to-peer log store

Scuttlebot

Scuttlebot is an open source peer-to-peer log store used as a database, identity provider, and messaging system. It has:

• Global replication
• File-syncronization
• End-to-end encryption

Scuttlebot behaves just like a Kappa Architecture DB. In the background, it syncs with known peers. Peers do not have to be trusted, and can share logs and files on behalf of other peers, as each log is an unforgeable append-only message feed. This means Scuttlebots comprise a global gossip-protocol mesh without any host dependencies.

Join us in #scuttlebutt on freenode.

Example Usage

# In bash:

# publish a message
sbot publish --type post --text "My First Post!"

# stream all messages in all feeds, ordered by publish time
sbot feed

# stream all messages in all feeds, ordered by receive time
sbot log

# stream all messages by one feed, ordered by sequence number
sbot hist $FEED_ID

// In javascript:

var pull = require('pull-stream')
var ssbClient = require('ssb-client')

// create a scuttlebot client using default settings
// (server at localhost:8080, using key found at ~/.ssb/secret)
ssbClient(function (err, sbot) {
  if (err) throw err

  // publish a message
  sbot.publish({ type: 'post', text: 'My First Post!' }, function (err, msg) {
    // msg.key           == hash(msg.value)
    // msg.value.author  == your id
    // msg.value.content == { type: 'post', text: 'My First Post!' }
    // ...
  })

  // stream all messages in all feeds, ordered by publish time
  pull(
    sbot.createFeedStream(),
    pull.collect(function (err, msgs) {
      // msgs[0].key == hash(msgs[0].value)
      // msgs[0].value...
    })
  )

  // stream all messages in all feeds, ordered by receive time
  pull(
    sbot.createLogStream(),
    pull.collect(function (err, msgs) {
      // msgs[0].key == hash(msgs[0].value)
      // msgs[0].value...
    })
  )

  // stream all messages by one feed, ordered by sequence number
  pull(
    sbot.createHistoryStream(feedId),
    pull.collect(function (err, msgs) {
      // msgs[0].key == hash(msgs[0].value)
      // msgs[0].value...
    })
  )
})

Use-cases

Scuttlebot's message-based data structure makes it ideal for mail and forum applications (see Patchwork). However, it is sufficiently general to be used to build:

• Office tools (calendars, document-sharing, tasklists)
• Wikis
• Package managers

Because Scuttlebot doesn't depend on hosts, its users can synchronize over WiFi or any other connective medium, making it great for Sneakernets.

Scuttlebot is eventually-consistent with peers, and requires exterior coordination to create strictly-ordered transactions. Therefore, by itself, it would probably make a poor choice for implementing a crypto-currency. (We get asked that a lot.)

---

Getting Started

• Install - Setup instructions
• Tutorial - Primer on developing with Scuttlebot
• API / CLI Reference

Key Concepts

• Secure Scuttlebutt, Scuttlebot's global database protocol
• Content Hash Linking
• Secret Handshake, Scuttlebot's transport-layer security protocol
• Private Box, Scuttlebot's end-to-end security protocol
• Frequently Asked Questions

Further Reading

• Design Challenge: Avoid Centralization and Singletons
• Design Challenge: Sybil Attacks
• Using Trust in Open Networks