taceywong / sf

Stream Framework is a Go library/package, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis.

SF:Stream Framework

---

Activity Streams & Newsfeeds

SF(Stream Framework) is a Go library/package, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. This is a clone of https://github.com/tschellenbach/Stream-Framework

Examples of what you can build are:

• Activity streams such as seen on Github
• A Twitter style newsfeed
• A feed like Instagram/ Pinterest
• Facebook style newsfeeds
• A notification system

(Feeds are also commonly called: Activity Streams, activity feeds, news streams.)

Background Articles

A lot has been written about the best approaches to building feed based systems. Here's a collection of some of the talks:

• Twitter 2013: Redis based, database fallback, very similar to Fashiolista's old approach.
• Etsy feed scaling: Gearman, separate scoring and aggregation steps, rollups - aggregation part two
• LinkedIn ranked feeds
• Facebook history
• Django project with good naming conventions
• Activity stream specification
• Quora post on best practices
• Quora scaling a social network feed
• Redis ruby example
• FriendFeed approach
• Thoonk setup
• Yahoo Research Paper
• Twitter’s approach
• Cassandra at Instagram
• Relevancy at Etsy
• Zite architecture overview
• Ranked feeds with ES
• Riak at Xing - by Dr. Stefan Kaes & Sebastian Röbke
• Riak and Scala at Yammer

Stream Framework

Installation

Installation through go get :

$ go get -u github.com/TaceyWong/sf

By default sf does not install the required dependencies for redis and cassandra:

Install stream-framework with Redis dependencies

$ 需要版本控制

Install stream-framework with Cassandra dependencies

$ 需要版本控制

Install stream-framework with both Redis and Cassandra dependencies

$ 需要版本控制

Authors & Contributors

Resources

Example application

Tutorials

Using SF:Stream Framework

Features

SF:Stream Framework uses Celery and Redis/Cassandra to build a system with heavy writes and extremely light reads. It features:

• Asynchronous tasks (All the heavy lifting happens in the background, your users don't wait for it)
• Reusable components (You will need to make tradeoffs based on your use cases, Stream Framework doesn't get in your way)
• Full Cassandra and Redis support
• The Cassandra storage uses the new CQL3 and Python-Driver packages, which give you access to the latest Cassandra features.
• Build for the extremely performant Cassandra 2.1. 2.2 and 3.3 also pass the test suite, but no production experience.