Comments (8)
That clearly biases the measurement since the information is lost (together with @guillermo-carrasco we saw some strange numbers after this tweak).
I guess the next step would be reproducing the error (in tests?), which might prove tricky. @fabianp, we are having this issue while we run this particular test:
https://github.com/SciLifeLab/facs/blob/master/tests/test_fastqscreen.py#L112
If you see something clearly wrong in the way we call it, please, let us know.
from memory_profiler.
@guillermo-carrasco I guess you meant OSError
as the exception, since:
File "/home/roman/.pyenv/versions/2.7.6/lib/python2.7/site-packages/memory_profiler.py", line 53, in _get_memory
except NoSuchProcess:
NameError: global name 'NoSuchProcess' is not defined
from memory_profiler.
Not really, I meant NoSuchProcess, you have to import it from psutil
from memory_profiler.
This just happened to me on a profiling run. Seeing how the uncaught exception causes the profiling to stop completely, I'd prefer to just miss out on the short-running job's memory usage.
from memory_profiler.
I pushed a fix in a9e5a02 . Feel free to reopen/send a pull request if it doesn't solve the issue.
from memory_profiler.
Didn't try this yet, but it looks like that won't work. I'm clearly getting a psutil.NoSuchProcess: psutil.NoSuchProcess process no longer exists (pid=1363)
, not an OSError
. But I can try a similar patch.
from memory_profiler.
from memory_profiler.
I ran into a similar problem on macOS, which I assume is also caused by this issue. I'm trying to track a process which creates tens or hundreds of children which live for a very short time, and occasionally I get the following error:
Traceback (most recent call last):
File "/Users/sasa/work/proj/env/bin/mprof", line 554, in <module>
actions[get_action()]()
File "/Users/sasa/work/proj/env/bin/mprof", line 244, in run_action
multiprocess=options.multiprocess, stream=f)
File "/Users/sasa/work/proj/env/lib/python2.7/site-packages/memory_profiler.py", line 358, in memory_usage
for idx, chldmem in enumerate(_get_child_memory(proc.pid)):
File "/Users/sasa/work/proj/env/lib/python2.7/site-packages/memory_profiler.py", line 115, in _get_child_memory
yield getattr(child, meminfo_attr)()[0] / _TWO_20
File "/Users/sasa/work/proj/env/lib/python2.7/site-packages/psutil/_common.py", line 336, in wrapper
return fun(self)
File "/Users/sasa/work/proj/env/lib/python2.7/site-packages/psutil/__init__.py", line 1119, in memory_info
return self._proc.memory_info()
File "/Users/sasa/work/proj/env/lib/python2.7/site-packages/psutil/_psosx.py", line 295, in wrapper
return fun(self, *args, **kwargs)
File "/Users/sasa/work/proj/env/lib/python2.7/site-packages/psutil/_psosx.py", line 421, in memory_info
rawtuple = self._get_pidtaskinfo()
File "/Users/sasa/work/proj/env/lib/python2.7/site-packages/psutil/_common.py", line 336, in wrapper
return fun(self)
File "/Users/sasa/work/proj/env/lib/python2.7/site-packages/psutil/_psosx.py", line 352, in _get_pidtaskinfo
ret = cext.proc_pidtaskinfo_oneshot(self.pid)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File "/Users/sasa/work/proj/env/lib/python2.7/site-packages/psutil/_psosx.py", line 326, in catch_zombie
raise AccessDenied(proc.pid, proc._name)
psutil.AccessDenied: psutil.AccessDenied (pid=43916)
I'll send a PR to ignore psutil.AccessDenied
in addition to psutil.NoSuchProcess
shortly.
from memory_profiler.
Related Issues (20)
- How to test memory usage for a time-loop function?
- Inconsistent result between line profiler and plot
- The @profile decorator will profile another decorator when chaining multiple decorators into a function. HOT 1
- include-children in decorator
- do not work in python 3.11 HOT 3
- Is maintained or not? HOT 4
- [BUG] invalid non-printable character U+FEFF HOT 6
- After delete the snapshot file, the function cannot be used when the memory profiler is opened again
- mprof: error: unrecognized arguments: --backend
- AttributeError: module 'pylab' has no attribute 'ioff' (Python 3.9) HOT 2
- Adding "--" to argment list may change the argument of the program started by `mprof run`. HOT 2
- non-ascii source characters misprocessed (maybe through 8859?) HOT 3
- mem usage and increment print is missing HOT 2
- I'm a beginner in the world of developing, I just finished learning python and html and css what's the next step please?I need a help with that. HOT 1
- Is it possible to compare two plots? HOT 1
- negative memory increment
- Is there a specific reason memory_usage return None when stream=True ? HOT 1
- Inconsistent documentation of units
- not sure how to interpret these results
- `tracemalloc` `backend` broken for `memory_usage`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from memory_profiler.