lmatteis / peer-tweet

Decentralized feeds using BitTorrent's DHT. Idea from Arvid and The_8472 "DHT RSS feeds" http://libtorrent.org/dht_rss.html

PeerTweet

Decentralized feeds using BitTorrent's DHT. Idea from Arvid and The_8472 "DHT RSS feeds" http://libtorrent.org/dht_rss.html

Screenshot

Quick links

• Download the latest binaries from the releases page.
• Find and share PeerTweet addresses from the addresses issues page.

What is PeerTweet?

BitTorrent's DHT is probably one of the most resilient and censorship-resistant networks on the internet. PeerTweet uses this network to allow users to broadcast tweets to anyone who is listening. When you start PeerTweet, it generates a hash @33cwte8iwWn7uhtj9MKCs4q5Ax7B which is similar to your Twitter username (ex. @lmatteis). The difference is that you have entire control over what can be posted because only you own the private key associated with such address. Furthermore, thanks to the DHT, what you post cannot be stopped by any government or institution.

Once you find other PeerTweet addresses you trust (and are not spam), you can follow them. This configures your client to store this user's tweets and broadcasts them to the DHT every once in a while to keep their feed alive. This cooperation of following accounts, allows for feeds to stay alive in the DHT network. The PeerTweet protocol also publishes your actions such as I just followed @919c.. or I just liked @9139.. and I just retweeted @5789... This allows the possibility for new users to find other addresses they can trust; if I trust the user @6749.. and they're following @9801.., then perhaps I can mark @9801.. as not spam. This idea of publicly tweeting about your actions also allows for powerful future crawling analysis of this social graph.

How does it work?

PeerTweet follows most of the implementation guidelines provided by the DHT RSS feed proposal http://libtorrent.org/dht_rss.html. We implemented it on top of the current BEP44 proposal which provides get() and put() functionality over the DHT network. This means that, rather than only using the DHT to announce which torrents one is currently downloading, we can use it to also put and get small amounts of data (roughly 1000 bytes).

PeerTweet differentiates between two types of items:

1. Your feed head. Which is the only mutable item of your feed, and is what your followers use to download your items and find updates. Your head's hash is what your followers use to know about updates - it's your identity and can be used to let others know about your feed (similar to your @lmattes handle). The feed head is roughly structured as follows:

{
  "d": <unsigned int of minutes passed from epoch until head was modified>,
  "next": <up to 80 bytes of the next 4 items in the feed, directly 1,2,3 and 4 hops away. 20 bytes each.>,
  "n": <utf8 name of the feed>,
  "a": <utf8 http url of an image to render as your avatar>,
  "i": <utf8 description of your feed>
}

2. Your feed items. These are immutable items which contain your actual tweets and are structured:

{
  "d": <unsigned int of minutes passed from epoch until when item was created>,
  "next": <up to 80 bytes of the next 4 items in the feed, 1, 2, 4 and 8 hops away. 20 bytes each.>,
  "t": <utf8 contents of your tweet. up to 140 chars>
}

Skip lists

The reason items have multiple pointers to other items in the list is to allow for parallel lookups. Our skip list implementation differs from regular implementations and is targeted for network lookups, where each item contains 4 pointers so that when we receive an item, we can issue 4 get() requests in parallel to other items in the list. This is crucial for accessing user's feeds in a timely manner because DHT lookups have unpredictable response times.

Following

When you follow someone, you're essentially informing your client to download their feed and republish it every so often. The DHT network is not a persistent one, and items quickly drop out of the network after roughly 30 minutes. In order to keep things alive, having many followers is crucial for the uptime of your feed. Otherwise you can still have a server somewhere running 24/7 which keeps your feed alive by republishing items every 30 minutes.

Install

Install dependencies.

$ npm install

Installing native modules

The app comes with some native bindings. I used this code to make it run on my computer:

Source: https://github.com/atom/electron/blob/master/docs/tutorial/using-native-node-modules.md

npm install --save-dev electron-rebuild

# Every time you run "npm install", run this
./node_modules/.bin/electron-rebuild

# On Windows if you have trouble, try:
.\node_modules\.bin\electron-rebuild.cmd

To get ed25519-supercop to work on Windows I also had to install node-gyp and all the Python2.7 and Visual Studio stuff which node-gyp requires: https://github.com/nodejs/node-gyp

Then run these commands to build it on Windows:

npm install -g node-gyp
cd ./node_modules/ed25519-supercop/
HOME=~/.electron-gyp node-gyp rebuild --target=0.36.9 --arch=x64 --dist-url=https://atom.io/download/atom-shell

Run

Run this two commands simultaneously in different console tabs.

$ npm run hot-server
$ npm run start-hot

Note: requires a node version >= 4 and an npm version >= 2.

Toggle Chrome DevTools

• OS X: Cmd Alt I or F12
• Linux: Ctrl Shift I or F12
• Windows: Ctrl Shift I or F12

See electron-debug for more information.

Toggle Redux DevTools

• All platforms: Ctrl+H

See redux-devtools-dock-monitor for more information.

Externals

If you use any 3rd party libraries which can't be built with webpack, you must list them in your webpack.config.base.js：

externals: [
  // put your node 3rd party libraries which can't be built with webpack here (mysql, mongodb, and so on..)
]

You can find those lines in the file.

CSS Modules support

Import css file as css-modules using .module.css.

Package

$ npm run package

To package apps for all platforms:

$ npm run package-all

Options

• --name, -n: Application name (default: ElectronReact)
• --version, -v: Electron version (default: latest version)
• --asar, -a: asar support (default: false)
• --icon, -i: Application icon
• --all: pack for all platforms

Use electron-packager to pack your app with --all options for darwin (osx), linux and win32 (windows) platform. After build, you will find them in release folder. Otherwise, you will only find one for your os.

test, tools, release folder and devDependencies in package.json will be ignored by default.

Default Ignore modules

We add some module's peerDependencies to ignore option as default for application size reduction.

• babel-core is required by babel-loader and its size is ~19 MB
• node-libs-browser is required by webpack and its size is ~3MB.

Note: If you want to use any above modules in runtime, for example: require('babel/register'), you should move them form devDependencies to dependencies.

Building windows apps from non-windows platforms

Please checkout Building windows apps from non-windows platforms.

Native-like UI

If you want to have native-like User Interface (OS X El Capitan and Windows 10), react-desktop may perfect suit for you.

Maintainers

This is a fork of the https://github.com/chentsulin/electron-react-boilerplate project.

License

MIT

peer-tweet's Issues

Uncaught SyntaxError: Unexpected token u

Trying to tweet and getting this error:

Security claims on BT Mainline DHT

(serious, please correct me if there is something I overlooked)

BitTorrent's DHT is probably one of the most resilient and censorship-resistant networks on the internet.

What do you base that claim on? As far as I know, there is an attack vector on the Mainline DHT that has been published for years[0] and that is being exploited in the wild[1]. As far as I can tell, the only reason the Mainline DHT still works is that nobody wanted that badly to damage it.

[0] Wang, Kangasharju: Real-world sybil attacks in BitTorrent mainline DHT; DOI:10.1109/GLOCOM.2012.6503215

[1] Wang, Kangasharju: Measuring Large-Scale Distributed Systems: Case of BitTorrent Mainline DHT; DOI:10.1109/P2P.2013.6688697

Use localforage by Mozilla instead of localStorage to avoid lags

localStorage is synchronous meaning it might lag if large lists are queried. https://github.com/mozilla/localForage provides the sami API as localstorage, but async, and fallbacks to indexeddb which is presumably faster.

Addresses

If you're using PeerTweet you can share your address using this page. To kick things off here's my address @2LuvbFkprVtmrkTjq3Ch1hNrBcse

Not compatible with your operating system or architecture

I got errors when trying to fire up npm install:
$npm install [email protected] postinstall /home/user/peer-tweet-master node node_modules/fbjs-scripts/node/check-dev-engines.js package.json npm WARN optional Skipping failed optional dependency /fbjs-scripts/babel/chokidar/fsevents: npm WARN notsup Not compatible with your operating system or architecture: [email protected] npm WARN optional Skipping failed optional dependency /webpack/watchpack/chokidar/fsevents: npm WARN notsup Not compatible with your operating system or architecture: [email protected]

OS: Fedora release 23 x86_64
npm: v3.6.0
node: v5.6.0

Followers should be a separate linked list

First of all, awesome ideas!

So, when you follow someone, it is published as part of the tweets linked list, this feels odd for a couple reasons:

• If you follow someone, and then publish a lot of tweets, you have to crawl a lot until you can discover who you follow to see their latest tweets
• When somebody else is crawling your tweets to read in their timelines, they don't care about who you follow, so it's unnecessary steps

My suggestion would be having two linked lists on the head instead of one: the tweets list and the following list.

The DhtDownload code branches out followers tweets anyway, it should even make this code simpler.

deprecated dependencies

These pop up when I run npm install. Just letting you know.

npm WARN deprecated [email protected]: graceful-fs version 3 and before will fail on newer node releases. Please update to graceful-fs@^4.0.0 as soon as possible.
npm WARN deprecated [email protected]: this package has been reintegrated into npm and is now out of date with respect to npm
npm WARN deprecated [email protected]: graceful-fs version 3 and before will fail on newer node releases. Please update to graceful-fs@^4.0.0 as soon as possible.
npm WARN prefer global [email protected] should be installed with -g
npm WARN prefer global [email protected] should be installed with -g

Cannot get Peer-Tweet running

Everything's in the title.
I followed the instructions on installing. I ran

npm install
npm install --save-dev electron-rebuild
./node_modules/.bin/electron-rebuild

After launching with 2 separate instance of terminal with these commands

npm run hot-server
npm run start-hot

When I start the second command line, I get this output

> [email protected] start-hot /home/l4p1n/peer-tweet
> cross-env HOT=1 NODE_ENV=development electron ./

(electron) companyName is now a required option to crashReporter.start
Error opening app
The app provided is not a valid Electron app, please read the docs on how to write one:
https://github.com/atom/electron/tree/v0.36.12/docs

Error: Cannot find module 'electron-debug'

npm ERR! Linux 4.2.0-36-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "run" "start-hot"
npm ERR! node v4.4.4
npm ERR! npm  v3.8.9
npm ERR! code ELIFECYCLE
npm ERR! [email protected] start-hot: `cross-env HOT=1 NODE_ENV=development electron ./`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] start-hot script 'cross-env HOT=1 NODE_ENV=development electron ./'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the PeerTweet package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     cross-env HOT=1 NODE_ENV=development electron ./
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs PeerTweet
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls PeerTweet
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /home/l4p1n/peer-tweet/npm-debug.log

And here is the content of /home/l4p1n/peer-tweet/npm-debug.log

0 info it worked if it ends with ok
1 verbose cli [ '/usr/bin/nodejs', '/usr/bin/npm', 'run', 'start-hot' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prestart-hot', 'start-hot', 'poststart-hot' ]
5 info lifecycle [email protected]~prestart-hot: [email protected]
6 silly lifecycle [email protected]~prestart-hot: no script for prestart-hot, continuing
7 info lifecycle [email protected]~start-hot: [email protected]
8 verbose lifecycle [email protected]~start-hot: unsafe-perm in lifecycle true
9 verbose lifecycle [email protected]~start-hot: PATH: /usr/lib/node_modules/npm/bin/node-gyp-bin:/home/l4p1n/peer-tweet/node_modules/.bin:/usr/bin:/home/l4p1n/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
10 verbose lifecycle [email protected]~start-hot: CWD: /home/l4p1n/peer-tweet
11 silly lifecycle [email protected]~start-hot: Args: [ '-c', 'cross-env HOT=1 NODE_ENV=development electron ./' ]
12 silly lifecycle [email protected]~start-hot: Returned: code: 1  signal: null
13 info lifecycle [email protected]~start-hot: Failed to exec start-hot script
14 verbose stack Error: [email protected] start-hot: `cross-env HOT=1 NODE_ENV=development electron ./`
14 verbose stack Exit status 1
14 verbose stack     at EventEmitter.<anonymous> (/usr/lib/node_modules/npm/lib/utils/lifecycle.js:245:16)
14 verbose stack     at emitTwo (events.js:87:13)
14 verbose stack     at EventEmitter.emit (events.js:172:7)
14 verbose stack     at ChildProcess.<anonymous> (/usr/lib/node_modules/npm/lib/utils/spawn.js:24:14)
14 verbose stack     at emitTwo (events.js:87:13)
14 verbose stack     at ChildProcess.emit (events.js:172:7)
14 verbose stack     at maybeClose (internal/child_process.js:827:16)
14 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)
15 verbose pkgid [email protected]
16 verbose cwd /home/l4p1n/peer-tweet
17 error Linux 4.2.0-36-generic
18 error argv "/usr/bin/nodejs" "/usr/bin/npm" "run" "start-hot"
19 error node v4.4.4
20 error npm  v3.8.9
21 error code ELIFECYCLE
22 error [email protected] start-hot: `cross-env HOT=1 NODE_ENV=development electron ./`
22 error Exit status 1
23 error Failed at the [email protected] start-hot script 'cross-env HOT=1 NODE_ENV=development electron ./'.
23 error Make sure you have the latest version of node.js and npm installed.
23 error If you do, this is most likely a problem with the PeerTweet package,
23 error not with npm itself.
23 error Tell the author that this fails on your system:
23 error     cross-env HOT=1 NODE_ENV=development electron ./
23 error You can get information on how to open an issue for this project with:
23 error     npm bugs PeerTweet
23 error Or if that isn't available, you can get their info via:
23 error     npm owner ls PeerTweet
23 error There is likely additional logging output above.
24 verbose exit [ 1, true ]

I guess I have to run npm install --dev but I prefer to ask to make sure.

build releases in github releases?

for users without nodejs installed,
why don't host the build releases in github releases?

https://help.github.com/articles/creating-releases/