Comments (3)
Some observations:
- The problem persists between restarts of the NVML-using program.
- If there is only a single compute process running, and I restart it, then the problem temporarily disappears.
- Passing in a timestamp makes it far more likely to break. If I always pass in None, then it'll usually keep working for half a minute or so, polling every 2s.
- nvtop doesn't have the issue. What do they do differently?
- Feels like a driver bug. Does this happen on every GPU?
from nvml-wrapper.
Here's the relevant nvtop code. Looks pretty different: https://github.com/Syllo/nvtop/blob/0316ce19581c3d8543cf6aa312d1569c56ca754f/src/extract_gpuinfo_nvidia.c#L761
from nvml-wrapper.
Another observation: Processes appear to only be returned if they are running. An idle process doesn't end up in the array, unless it was non-idle very recently. This accounts for what happens if I set the timestamp -- it reduces the horizon.
Also means that swallowing the error (and returning []) should be a valid workaround.
from nvml-wrapper.
Related Issues (20)
- Load NVML lib at runtime HOT 4
- Any way to build without /usr/lib? HOT 4
- Use the `#[doc(alias = "...")]` attribute on wrapper methods
- Rename `NVML` struct to `Nvml`
- Accept more variants of lib name or make internal init public HOT 2
- Lifecycle problems with `device_by_index` HOT 3
- Explore reference counting to maintain lifetime relationships among NVML data structures
- nvmlDeviceSetGpuLockedClocks vs nvmlDeviceSetMemoryLockedClocks HOT 2
- GPU Temp sensors HOT 2
- feature request: add support for `nvmlDeviceGetGraphicsRunningProcesses_v2` HOT 5
- Device Brand function fails for new RTX cards
- Cannot call legacy functions HOT 3
- "libnvidia-ml.so: cannot open shared object file: No such file or directory HOT 3
- Segmentation fault when fetching graphics processes HOT 6
- FailedToLoadSymbol("GetProcAddress failed") with Windows studio driver v536.99 HOT 2
- failing to load driver nvml from wsl2 HOT 3
- Support for querying scoped fields HOT 2
- Update memory info to v2
- Fan Control Wrappers HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nvml-wrapper.