wegiangb / jupyterstream

Data Deadline? Jupyter to the Rescue! - Blog Post

jupyterstream

Python procedures streamlined by handy manipulations of Jupyter notebooks
This code accompanies our blog which discusses the benefits of usage of runipy, parallelising executions and aggreating the results in one master notebook using nbformat.

In detail we discuss:

• Applying runipy, a convienient tool to run a notebook from the command line
• Parallelising executions the bash command parallel.
• Using nbformat, a python module to read text and image outputs from notebooks.

We suggest using runipy to loop over notebooks or to run in parallel. Once the notebooks runs are completed, to figure out which runs were succesful and which require special attention, a master notebook could collect the important metrics and figures (!) for analysis.

Disclaimer: This is not an exhaustive explanation of the functionality of runipy, nbformat or parallel, but rather a suggestion from which one can develop their own preference of use.

Setup

python setup.py install 

or

python setup.py develop

Demo

See the demo folder for an example demonstration.