Coder Social home page Coder Social logo

Comments (5)

inducer avatar inducer commented on June 1, 2024

Can something like valgrind maybe provide details on where those allocations are taking place?

from pyopencl.

kif avatar kif commented on June 1, 2024

Here are the valgrind "massif" profile for two calls of the program when running on a limited number of images (2000), with and without profiling activated. Valgrind still suggests to look at h5py rather than pyopencl but the triggering of the option makes 16G difference in memory consumption.

Without profiling:
image

With profiling:
image

from pyopencl.

kif avatar kif commented on June 1, 2024

I run it several other times and it looks like the profiling in OpenCL prevents the memory from being freed.

from pyopencl.

kif avatar kif commented on June 1, 2024

So I tired to collect only timestamps for each event instead of the complete process.
The patch is for now implemented in:
silx-kit/silx#3690

The memory profile looks like this, now. One would have expected 10 memory free (since 10 files are processed) but fewer are visible.

image

from pyopencl.

kif avatar kif commented on June 1, 2024

I got struck by something similar in another project ... but profiling was not involved this time.
https://github.com/kif/multianalyzer/blob/main/multianalyzer/opencl.py
The pattern was similar: read data from an HDF5 file with large chunks and send them to the GPU ...
But once again, unable to reproduce the behaviour within a self contained script.
Calling the pyopencl.array method finalize helps in freeing the memory on the CPU

from pyopencl.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.