verigak / progress Goto Github PK
View Code? Open in Web Editor NEWEasy to use progress bars for Python
License: ISC License
Easy to use progress bars for Python
License: ISC License
I did some reading online and it mentioned monkey patching. Any tips on how I can do that in this case?
Here's the complete traceback:
In [45]: spinner = Spinner('')
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-45-edb2ea8bc262> in <module>()
----> 1 spinner = Spinner('')
~/.local/lib/python3.6/site-packages/progress/helpers.py in __init__(self, message, **kwargs)
29 self.message = message
30
---> 31 if self.file.isatty():
32 if self.hide_cursor:
33 print(HIDE_CURSOR, end='', file=self.file)
simple
counter = Counter('waiting... ')
for _ in range(10):
counter.next()
time.sleep(1)
leaves my console without cursor, so it seems that counter.finish() is not run.
pg = Spinner('中文')
pg.finish()
Reading at the first example in the readme:
from progress.bar import Bar
bar = Bar('Processing', max=20)
for i in range(20):
# Do some work
bar.next()
bar.finish()
I notice that no indication of the current iteration, neither the total number of iteration of the loop is given as argument to the main bar.next()
function. How does it guess those two numbers?
I think that giving a rough idea of the mechanisms in place in the readme would be appreciated. It seems a bit magic, and there might be issues we can prevent if users know where the above mentioned information are coming from.
I've noticed that the progress bar sometimes blinks when clearing. I think it's caused by this line in WritelnMixin
:
print('\r\x1b[K', end='', file=self.file)
This control sequence introducer \x1b[K
is supposed to clear the characters to the right of the cursor, while \r
returns the cursor to the left.
At least in my case it is enough to use \r
without clearing anything. The characters will get overridden. Changing the line to print('\r', end='', file=self.file)
and I haven't noticed any side effects yet. But maybe I'm missing something obvious here?
So my suggestion would be to either simply use \r
for "clearing" the line, or to at least let the user customize this via some parameter.
Not printing a newly created progress bar gives flexibility, but it would be better if users were given a choice about this.
Using goto(0) does not work (unless self.backtrack is turned on) since the library computes a delta to decide whether or not to run the update() method. Even if this worked, it would be a hack. A more proper solution would be to add a show() or even better a start() method for this purpose (you have a finish() one implemented) .
P.S: If I were you I would also rename "max" attribute to "size". It is still monosyllable and it sounds better. I've already asked 2000 chicks about this. Of course this would break the code of millions of users including myself but I have been doing some rough calculations lately and I think I will survive. Even if I don't, please don't cry for me. The planet is overpopulated anyway! :-)
I know from Windows 95 not to put too much faith in ETAs, but this is consistently and obviously wrong! progress
1.4 will show an ETA of at most a few minutes, even when it is 50% of the way through a large hour-long process. Is there some timing module that it needs to work more reliably?
I am using it with requests
and humanize
to show a progress bar for HTTP PUT requests of large files:
class HumanBar(ChargingBar):
@property
def eta_human(self):
return humanize.naturaldelta(self.eta)
@property
def max_human(self):
return humanize.naturalsize(self.max, binary=True)
class MonitoredFile(object):
def __init__(self, handle, size=0, callback=None):
self.len = size
self.handle = handle
self.callback = callback
self.total = 0
self.bar = HumanBar('Uploading %(max_human)s: ', max=size, fill='>', suffix='%(percent)d%% ETA %(eta_human)s ')
def __len__(self):
return self.len
def read(self, *args):
chunk = self.handle.read(*args)
if len(chunk) > 0:
self.total += len(chunk)
self.bar.next(len(chunk))
if self.callback:
self.callback(len(chunk), self.total, self.len)
return chunk
The percentage is accurate, but the ETA isn't even close...
When "max" is set to zero (can happen if there's nothing to process) the code returns a ZeroDivision error when update() is called.
The culprit seems to be in progress/__init__.py
, like 156:
@property
def progress(self):
return min(1, self.index / self.max)
I couldn't find the usage of Stack progress bar, any implementations?
I am interested in having multiple progress bars work. Can someone give me a pointer on how to start on this? This is important when reporting the state of a pipeline which is doing a count of work at each stage but all stages are executing in parallel.
Here is an example of my current problem. I want to update both bars in one loop and then finalize them both using this code:
from time import sleep
from progress.bar import Bar
bar = Bar('Downloading', max=20)
bar2 = Bar('Processing', max=20)
for i in range(20):
sleep(0.2)
bar.next()
bar2.next()
bar.finish()
bar2.finish()
It produces this output:
Processing |################################| 20/20
You only are shown bar2 since it did it's update last. How can multiple bar support be added to this codebase?
The only impediment to reusing the same progress object multiple times is that index
is not reset when iter
is called -- doing that would make it very convenient to set the message based on a property of the instance, and then reuse it to track multiple operations.
Possibly, each time it is entered as a context manager, the same ... or a reset method, all would satisfy the same desire to reuse the class.
This is desirable to me because it lets me build various attributes (eg: message) early in the process, when I have all the data to hand, rather than later, when I don't. (I mean, I can surely save it and use it later, but ... adding attributes to configure a class later when I have them here and now feels less optimal, y'know?
The example for the bar shows using finish() when you're done with a progress item. However, I noticed spinner just leaves things hanging. The example doesn't use finish(). I tried it but it seems to be a no-op?
Thanks,
-Clint
Pip version: 9.0.1
I am trying to install pyxform package at runtime in QGIS 3. When i am running same code in cmd it runs fine. But when I runs in QGIS 3 python console it gives error.
import pip
pip.main(['install','pyxform','--user'])
Traceback (most recent call last):
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\basecommand.py", line 215, in main
status = self.run(options, args)
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\commands\install.py", line 335, in run
wb.build(autobuilding=True)
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\req\req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\req\req_set.py", line 620, in _prepare_file
session=self.session, hashes=hashes)
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\download.py", line 821, in unpack_url
hashes=hashes
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\download.py", line 659, in unpack_http_url
hashes)
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\download.py", line 882, in _download_http_url
_download_url(resp, link, content_file, hashes)
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\download.py", line 585, in _download_url
progress_indicator = DownloadProgressBar(max=total_length).iter
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\utils\ui.py", line 158, in __init__
super(WindowsMixin, self).__init__(*args, **kwargs)
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\utils\ui.py", line 82, in __init__
super(InterruptibleMixin, self).__init__(*args, **kwargs)
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\utils\ui.py", line 118, in __init__
super(DownloadProgressMixin, self).__init__(*args, **kwargs)
File "C:\PROGRA~1\QGIS3~1.0\apps\Python36\lib\site-packages\pip\_vendor\progress\helpers.py", line 58, in __init__
if self.file.isatty() and self.hide_cursor:
AttributeError: 'NoneType' object has no attribute 'isatty'
```
Hi
I know suffixes can include text, but I would like to append a variable to the suffix, so it is updated along with the percentage. Would that be feasible?
My use case is:
I have a progress bar which outputs the progress of a video transcode. I'd like to append the frames per second value and current output file size right after the progress percentage
Transcoding |█ | 3% 123.45 fps Current Size: 12345MB
I already have that variables which update regularly, but can't figure out how to make them appear in the suffix and be updated as the progress bar changes.
Is this even possible?
Thanks
Just like Bar
supports using %(propname)s
, Spinner
should do so as well. This allows each loop to give information on what is happening on each iteration.
spinner = Spinner('message with %(context)s')
while some_cond():
spinner.context = what
spinner.next()
This module doesn't seem to work using an iPython console.
Even simply running the example:
from progress.bar import Bar
bar = Bar('Processing', max=20)
for i in range(20):
# Do some work
bar.next()
bar.finish()
results in successful completion but with no progress bar.
Output:
In [1]: from progress.bar import Bar
...:
...: bar = Bar('Processing', max=20)
...: for i in range(20):
...: # Do some work
...: bar.next()
...:
...: bar.finish()
In [1]:
In [1]:
In [2]:
Hi,
The website / module docs use:
from progress.bar import Bar
Where the right import clause is nowadays:
from progress_bar import ProgressBar
I am interested in packaging this project. Would it be feasible to have a tagged release?
When a KeyboardInterrupt (or any other exception) is raised inside a loop over a iteratoor gotten from Bar().iter() the terminal is not reset. This means that the cursor is gone. A reset in the terminal fixes it.
$ python2 test_bar.py
Processing |###### | 19/99Traceback (most recent call last):
File "test_bar.py", line 8, in <module>
raise KeyboardInterrupt
KeyboardInterrupt
$ # No cursor in the terminal any more
test_bar.py:
import time
from progress.bar import Bar
for i in Bar('Processing').iter(range(1, 100)):
time.sleep(0.01)
if i == 20:
raise KeyboardInterrupt
Reproducer:
from progress import spinner, bar
s = spinner.Spinner('Foo ... ')
s.next()
s.finish()
There is no newline after this spinner, so my terminal prompt is on the same line, e.g.:
Foo ... \[bkabrda@zizalka test]$
I guess the problem is in WriteMixin (used by spinners), which doesn't print newline, as opposed to WritelnMixin (used by bars) which prints new line.
Before the hide_cursor option was introduced, finish() just printed a new line and calling the method after the progress was completed was optional. If the user wanted to overwrite the progress bar line, he could just omit calling finish() and print something like this "\r<new_message>\033[K" to the output.
Right now finish() does 2 jobs. It prints a new line and unhides the cursor if it was hidden. If the hide_cursor is enabled, not calling finish() is not an option anymore. It would be useful if the user could pass an optional argument to finish() to disable the "print new line" action.
for Bar class, it's useful showing elapsed time default.
thanks.
Please bundle test_progress.py in the pypi package so that downstream distributors can test during installation.
This is what I write:
bar = Bar(' fetching data', max=len(platforms)*TIMES_NUM)
for platform in platforms:
...some work here...
bar.next()
bar.finish()
This is what I get from progress==1.1 from PyPI:
b' fetching data |################################| 150/150'
The progress bar works correctly but its output is obviously bytes instead of str and therefore it is formatted not very nicely.
I believe this is the reason: https://github.com/verigak/progress/blob/master/progress/helpers.py#L40
if i want to get the current index, i need to write "counter" variable manually or "enumerate(list)".
Is there any other way?
thanks.
It doesn't seem to be showing on Jupyter lab for me. Is there a way to get it to work?
Is the package deprecated for newer versions of python?
Every call to update() causes the progress bar to produce output. If I'm using a Bar to track a loop of millions of iterations then my terminal is flooded with output even though the progress bar itself doesn't need to change.
Perhaps add an option lazy_write which, if true, causes writeln to remember what it last wrote - and if it's no different, it doesn't write anything to the terminal.
Hello, Your work is so good.
I tried to use this package in both Jupyter Notebook and in Colab, but I can't see any bars!
Am I doing something wrong here to get "?[K" added to the necessary string?
code :
bar = IncrementalBar('Cleaning text', max=len(links), suffix='%(percent)d%%' )
bar = IncrementalBar('Scraping Links', max=len(paper.articles), suffix='%(percent)d%%')
Output:
?[KScraping Links |████████████████████████████████| 100%
getting content
?[KCleaning text |████████████████████████████████| 100%
scraper_main
?[KScraping Links |████████████████████████████████| 100%
getting content
?[KCleaning text |████████████████████████████████| 100%
scraper_main
?[KScraping Links |████████████████████████████████| 100%
Since this module is visual, it would be great to show screenshots (or even animated gifs) to showcase what it does.
See https://github.com/Frozenball/pytest-sugar/blob/master/README.rst for some possible inspiration.
Shouldn't self.avg be (roughly) equal to self.elapsed/self.index? In my test (file uploading) it comes out differently:
1.578591763973236e-08 (self.avg) vs. 7.149365691934405e-08 (self.elapsed/self.index)
Eta is then not correct (less than actual).
Thanks!
Hi @verigak,
How do you integrate this library with urllib.request.urlretrieve?
In the image on README.md
you show a Random progress bar.
Are you open for PRs on this (i.e. a class for Random bar instead of users working around IncrementalBar
)?
I am using PyCharm community Version 2017.3 using a Python 3.6 virtual environment (created with Miniconda) on Windows 10.
When testing the example in the ReadME (to print the bar), nothing prints on the console.
Hi,
I've found that codes(1.4) downloaded from pypi (with pip install) are different from those in repo.
To be specific, it's lacking with
support.
$ virtualenv test_env
$ source test_env/bin/activate
$ pip install -U progress
$ cat test_env/lib/python3.7/site-packages/progress/__init__.py | grep enter
A version bump should solve that.
I recently tried using this library's SigIntMixin
to resolve pip issue #2462, but it turns out that SigIntMixin
does not provide quite enough functionality to work cleanly in that context.
For pip, I ended up implementing a very similar InterruptibleMixin
class: see its docstring for discussion.
The primary differences between the two are that InterruptibleMixin
:
SystemExit
This means that InterruptibleMixin
should cooperate with any SIGINT handler the calling context already has in place: in particular, it cooperates with Python's default handler and propagates its KeyboardInterrupt
, which pip's existing code handles more easily. (Other programs could catch and recover from it, to let the user cancel a download and continue with something else, for example.)
With all the above said, I am filing this issue mainly to spark discussion: although something like InterruptibleMixin
works better for pip and maybe other programs, I am not sure that it is the best way of dealing with this problem in general.
The primary need for a mixin like this is simply to reliably call contextual cleanup code (like the cursor hiding and unhiding mixins), which Python already has a more robust solution for in the form of context managers.
If all the progress bar display setup and cleanup operations were implemented in terms of the context manager API, then there should be no need for comparative hacks like SIGINT handlers at all: the code would do the right thing for KeyboardInterrupt
as well as any other exceptions that cross the context boundary.
Hi,
I try the code in terminal, and it works very well. However, when I run the code test_progress.py
from PyCharm, nothing displays.
I cannot seem to get a rolling average to work with the Bar
class and using a suffix of suffix = suffix = "%(progress) %(avg)f %(percent).1f%% - %(eta)ds"
. I instead get %(avg)f
in the output.
Any ideas?
It would be nice if progress
supported the situation when sys.stderr
is changed at runtime. Currenly, it is bound in Infinity.file
at import time and further changes to sys.stderr
are not reflected. I'd like to suggest changing Infinity.file
into a property returning sys.stderr
.
Related: Drekin/win-unicode-console#11.
Hi,
the following code triggers a wrapping bug in connection with the terminal size:
import time, progress.bar
bar = progress.bar.Bar(
suffix = '%(index)d of %(max)d (%(percent)d%%) '
'eta: %(eta_td)s time elapsed: %(elapsed_td)s',
max = 100)
for index in range(100):
bar.next()
time.sleep(0.1)
bar.finish()
This is the output on a 87x40 terminal (the bug "starts" at value 10):
|### | 10 of 100 (10%) eta: 0:00:09 time elapsed: 0:00:0
|### | 11 of 100 (11%) eta: 0:00:09 time elapsed: 0:00:0
|### | 12 of 100 (12%) eta: 0:00:09 time elapsed: 0:00:0
|#### | 13 of 100 (13%) eta: 0:00:09 time elapsed: 0:00:0
|#### | 14 of 100 (14%) eta: 0:00:09 time elapsed: 0:00:0
|#### | 15 of 100 (15%) eta: 0:00:09 time elapsed: 0:00:0
|##### | 16 of 100 (16%) eta: 0:00:09 time elapsed: 0:00:0
|##### | 17 of 100 (17%) eta: 0:00:09 time elapsed: 0:00:0
|##### | 18 of 100 (18%) eta: 0:00:09 time elapsed: 0:00:0
|###### | 19 of 100 (19%) eta: 0:00:09 time elapsed: 0:00:0
|###### | 20 of 100 (20%) eta: 0:00:09 time elapsed: 0:00:0
|###### | 21 of 100 (21%) eta: 0:00:08 time elapsed: 0:00:0
|####### | 22 of 100 (22%) eta: 0:00:08 time elapsed: 0:00:0
|####### | 23 of 100 (23%) eta: 0:00:08 time elapsed: 0:00:0
|####### | 24 of 100 (24%) eta: 0:00:08 time elapsed: 0:00:0
|######## | 25 of 100 (25%) eta: 0:00:08 time elapsed: 0:00:0
|######## | 26 of 100 (26%) eta: 0:00:08 time elapsed: 0:00:0
|######## | 27 of 100 (27%) eta: 0:00:08 time elapsed: 0:00:0
|######## | 28 of 100 (28%) eta: 0:00:08 time elapsed: 0:00:0
|######### | 29 of 100 (28%) eta: 0:00:08 time elapsed: 0:00:0
|######### | 30 of 100 (30%) eta: 0:00:08 time elapsed: 0:00:0
|######### | 31 of 100 (31%) eta: 0:00:07 time elapsed: 0:00:0
3
[...]
This is the output on a maximized terminal:
|################################| 100 of 100 (100%) eta: 0:00:00 time elapsed: 0:00:09
Since progress hides the cursor, if the program exits/crashes prematurely or is interrupted via the keyboard, the cursor state is left in the hidden mode. Some attempt should be made to restore this state under most circumstances.
Python 3.6 has updated the console on Windows to print Unicode characters correctly and always use UTF-8 for sys.std[in|out|err].encoding
. Unfortunately, 6 of the characters in IncrementalBar.phases
do not have glyphs in the default console fonts, so the bar displays the missing glyph image (question mark in a box) while incrementing.
Programs (like pip
) that check stdout.encoding
is Unicode before using IncrementalBar
will now use it instead of the non-incremental variety. I made the tweak below in my copy of pip and the result was acceptable, if not especially "incremental" anymore. Considering the extra glyphs are probably not going to be added anytime soon, and are unlikely to make it onto existing versions of Windows anyway, I'd like to propose this change (with appropriate comments as you wish) to avoid an ugly bar.
class IncrementalBar(Bar):
if sys.platform.startswith('win'):
phases = (u' ', u'▌', u'█')
else:
phases = (u' ', u'▏', u'▎', u'▍', u'▌', u'▋', u'▊', u'▉', u'█')
In python 3.4, when I call finish() method on a progress bar object it results in NameError: name 'unicode' is not defined. However this is fine when used in python verison 2.x
Please fix the issue or suggest an alternative approach to avoid running into this issue.
if caller has set up a bar generator my_bar_gen, it should be able to:
my_bar_gen.next(10)
to advance the running progress bar by 10, provided it's faster than:
for i in range(10):
my_bar_gen.next()
I'm not using the conventional print to display information to my user and I would need to have the progress bar as a str variable. Is it possible ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.