medusagit / streamparse

streamparse lets you run Python code against real-time streams of data with Apache Storm.

Streamparse lets you run Python code against real-time streams of data via Apache Storm. With streamparse you can create Storm bolts and spouts in Python without having to write a single line of Java. It also provides handy CLI utilities for managing Storm clusters and projects.

The Storm/streamparse combo can be viewed as a more robust alternative to Python worker-and-queue systems, as might be built atop frameworks like Celery and RQ. It offers a way to do "real-time map/reduce style computation" against live streams of data. It can also be a powerful way to scale long-running, highly parallel Python processes in production.

Documentation

http://streamparse.readthedocs.org/en/latest/

User Group

Follow the project's progress, get involved, submit ideas and ask for help via our Google Group, [email protected].

Contributors

Alphabetical, by last name:

• Dan Blanchard (@dsblanch)
• Keith Bourgoin (@kbourgoin)
• Arturo Filastò (@hellais)
• Jeffrey Godwyll (@rey12rey)
• Daniel Hodges (@hodgesds)
• Wieland Hoffmann (@mineo)
• Tim Hopper (@tdhopper)
• Omer Katz (@thedrow)
• Aiyesha Ma (@Aiyesha)
• Andrew Montalenti (@amontalenti)
• Rohit Sankaran (@roadhead)
• Viktor Shlapakov (@vshlapakov)
• Mike Sukmanowsky (@msukmanowsky)
• Cody Wilbourn (@codywilbourn)
• Curtis Vogt (@omus)

Roadmap

See the Roadmap.