kuk / log-progress Goto Github PK

Soon I hope to release an instrument control library that I've been working on. It will include some basic jupyter notebook capability. The log_progress code here would be an ideal type of indicator for that use.

Could you post a LICENSE file so that I can redistribute (with credit of course), or to clarify that I cannot?

Currently, it's possible to do something like:

def worker(i):
    d = 0
    for _ in range(i):
        d += 1
    return d

data = [int(5e6) for _ in range(10)]

map(worker, log_progress(data, every=1))

If I try to do the same using a multiprocessing mapper, the progress bar is immediately completed.

from multiprocessing import Pool

def worker(i):
    d = 0
    for _ in range(i):
        d += 1
    return d

data = [int(5e6) for _ in range(10)]

p = Pool(2)
processes = p.map(worker, log_progress(data, every=1))
p.close()

Would it be possible to somehow make this work?

Hi. I am relatively new to jupyter/ipython, and this looks just like what I need in order to not get bored and turn off the calculation in themiddle. I was wondering if you could write a quick installation guide.

Thank you so much for sharing! This is pretty awesome! I would love to see this as some standard package somewhere so that I do not need to copy this code all the time :)

IPython 5.1.0

Widget Javascript not detected.  It may not be installed properly. Did you enable the widgetsnbextension? If not, then run "jupyter nbextension enable --py --sys-prefix widgetsnbextension"

I run

jupyter nbextension enable --py --sys-prefix widgetsnbextension

and rerun ipython console. But still

Widget Javascript not detected.  It may not be installed properly. Did you enable the widgetsnbextension? If not, then run "jupyter nbextension enable --py --sys-prefix widgetsnbextension"

say you've got a NumPy array [1,2,3...]
and a function func(x)

so you do: vecfunc=numpy.vectorize(func)
and then use the power of vectorization to:
vecfunc(x)

how would you hack this together?

Trying with iterrows to iterate through data frame rows:

for index, c in log_progress(df_search.iterrows(), every=1):

It doesn't seem to be able to identify the number of items.

I ran this code:

import time
from time import sleep

def check_url(url):
    sleep(0.1)

urls = range(100)

for url in log_progress(urls, every=1):
    check_url(url)
    
for index in log_progress(range(100), every=1):
#     print("testing")
    sleep(0.1)

and it resulted in:
VBox(children=(HTML(value=''), IntProgress(value=0)))

Am I missing something?
Running on Windows 10:
anaconda custom py36h363777c_0
anaconda-client 1.7.2 py36_0
anaconda-navigator 1.9.7 py36_0
anaconda-project 0.8.2 py36_0

Prints VBox(children=(HTML(value=''), IntProgress(value=0, max=360334)))

Server Information:
You are using Jupyter notebook.

The version of the notebook server is: 6.0.0
The server is running on this version of Python:
Python 3.7.3 | packaged by conda-forge | (default, Jul 1 2019, 21:52:21)
[GCC 7.3.0]

Current Kernel Information:
Python 3.7.3 | packaged by conda-forge | (default, Jul 1 2019, 21:52:21)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.11.1 -- An enhanced Interactive Python. Type '?' for help.

Are you considering releasing this on PyPi?

When I have N threads going through a list of steps, as in:
Parallel(n_jobs=N)(delayed(calculate_stuff)(i) for i in log_progress(steps))
The bar is always N steps ahead.

This is simply AMAZING.
Any idea if I can install it via pip/conda so I can import them?

Again, Thanks.

Errrneist
20181022.

When I run the code in jupyter lab. I got this print out:

VBox(children=(HTML(value=''), IntProgress(value=0, max=2)))

This is by far the best progress bar to use in Jupyter, so thank you! I am being greedy now, would it be possible to implement an ETA based on previous iterations as well as keep track of running time at a certain interval. Not looking to get too crazy, just a basic average of time between task start and end multiplied by the number of tasks left. Then for the running time, I would imagine updating every second might be unnecessary so maybe an argument with a default set to something like 5 seconds.

Thank you again for sharing!

First thanks a lot for the great work, I am using log_progress in a couple of Notebooks and it improves presentations to my customers a lot.

Currently I am shifting to Jupyter Lab and as I was running my Notebooks in Lab, I recognized, that the Progress bar isn't rendered only the VBox object text representation is appearing as a result

VBox(children=(HTML(value=''), IntProgress(value=0, max=84)))