Comments (25)
Haha I use TIFFs for such data all the time. They have some compression support, but I don't know anything about its performance in comparison to other options. TIFF also has vast metadata capabilities, although see my upcoming comment in #263 for caveats.
from imageio.
Do you have a link for these claims ? (I was personally not aware of it)
Is that something that cannot be fixed upstream, instead of re-inventing a new format for data serialization ?
from imageio.
I tried NPZ in pypy and it simply was not implemented. In Vispy we've had some trouble with data stored in npz files. It seemed that with a certain combination of Python2 and Numpy the file could not be read in Python3.
The title could also be: implement plugins for other (existing) 3D data formats.
from imageio.
Do you have specific formats in mind ?
from imageio.
In particular MHD (#29) and hdf5. However, the first needs 2 files to store data (bah) and I suspect hdf5 wont work without relying on another library ...
from imageio.
Actually, dual-file data storage is still pretty common. Most of the proprietary formats I deal with in my research are designed that way (one file for the descriptor, one for the raw data). So I'd be interested to see how you are planning to wrap that out.
For HDF5, which I am also familiar with, pytables and h5py are already decent solutions around libhdf5. I don't know whether there is a niche for a pure-Python implementation of it though.
from imageio.
For HDF5, just stick with h5py...
from imageio.
About npz: it looks like there are no problems when files are created in Python 3 (see this issue). Is it conceivable to just throw a big warning when users save npz files in Python 2? Python 3 users will probably never have a problem. It might just be a matter of mentioning this potential problem in the documentation.
from imageio.
As for hdf5, imageio could provide a thin wrapper to easy the storing and retrieving of image/volume data. Not sure if this is at all useful; it might already be simple enough with pytables/h5py.
Still, I'd like to have something that is pure Python (i.e. works everywhere) and does not rely on another lib...
from imageio.
I agree that a pure Python HDF5 lib might be useful generally speaking, but it might be a lot of work. In my experience the HDF5 C API is horrible to work on, even just getting the data can be quite complicated...
Also, anyone willing to work with HDF5 files in Python will always have h5py or pytables installed. (they are installed by default in anaconda for example)
from imageio.
I agree, but I was not talking of hdf5 per see.
from imageio.
@almarklein then I think you have too options:
- npz/npy
- flat binary + metadata (either in a header, or in a second file)
from imageio.
npz is not widely available (it depends on numpy and is not available on pypy). I think I can do better than a flat binary with meta data :) Would be best if there was already a format that we could use. If not, I might just implement something simple.
from imageio.
Oh so you really mean pure Python (no NumPy etc)? Just out of curiosity, why do you need PyPy support?
How can you do best and simpler than flat+header in pure Python?
from imageio.
I just hate it that we cannot do volumes in pypy :)
from imageio.
Reopening. I'm thinking of something pure Python (not relying on simpleITK), or more generic (preferably both).
from imageio.
Have you looked into the formats that OpenImageIO supports for inspiration?
from imageio.
I did now. But their plugins that support volumetric images are either based on hdf5 or aimed very much on animation.
from imageio.
There's the nrrd format, which is quite simple (basically flat binary, optionally compressed): http://teem.sourceforge.net/nrrd/
I've used the pure python (with numpy) implementation here: https://github.com/mhe/pynrrd/blob/master/nrrd.py
from imageio.
Thanks @dimatura that sounds interesting, especially since its pure Python
from imageio.
@almarklein why has TIFF not been discussed here? iirc TiffFile has a pure python implementation...?
from imageio.
Would TIFF be suited for storing a 512x512x512 volume? I've never seen people do that with GIF, perhaps because the compression is not that good?
In some cases it would also be necessary to store meta data like spacing between voxels, the origin, or a transformation matrix.
from imageio.
Fair enough :) The Tiff format recently became 3D capable by supporting volread()
. My impression was that this was mostly to read all channels at once, but it can indeed also be used to store actual volumetric data.
from imageio.
Maybe this is why Tiff, although 3D capable, is a bit hard to sell as the format for 3D data: #263 (comment) :)
from imageio.
🏳️
from imageio.
Related Issues (20)
- ImageResource type should support bytes HOT 1
- [Bug Report] imageio v3 crashed when read hdr images after a certain number of times
- Link to the documentation in the README is broken HOT 1
- imread() fails on HTTPS, code example from documentation HOT 3
- 8bit grayscale now loaded as RGB HOT 5
- 2.31.4: documentation build fails HOT 1
- iio pyav plugin crash along with python when handling a webm file, setting format to rgba HOT 2
- It does not seem to support avif image format HOT 1
- Support for Pillow v10.1 (re: setting `image.mode`) HOT 8
- Inconsistency when reading floating point .exr image file HOT 3
- imageio.imread changes the values of graycale pixels, try to read 12-bit png grayscale image. HOT 8
- super slow video reads HOT 6
- imageio v2.33.0 requieres Pillow>=9.1.0 for GIFs HOT 1
- Trouble using freeimage plugin on MacOS arm64 HOT 2
- 2.33.1: pytest is failing in 5 units HOT 1
- [Question] What tonemapping method does the imageio library use when reading HDR images? HOT 2
- title of the gif got stacked together HOT 2
- UnicodeDecodeError if metadata is not UTF-8 encoded (pyav plugin) HOT 3
- [request] more color/palette control on GIF writing HOT 3
- Transition from v2 ffmpeg read/write to v3 pyav read/write: quality, batching HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from imageio.