_______ __
| | |.-----.|__|.-----.-----.
| || _ || || _ | -__|
|___|___|| __||__|| __|_____|
|__| |__|
Hpipe
Hpipe is a workflow engine supporting hybrid workflows with built-in support for Hadoop Streaming and Hive.
It is useful in many production scenarios such as offline data processing of click-through-rate prediction for online advertising.
Features
- Integrate various types of jobs into a uniformed workflow.
- Job status tracking mechanism: keep job status and auto-resume from breakpoints.
- Workflow descriptive language based on Graphviz being easy to edit and visualize.