imi-bigpicture / opentile Goto Github PK
View Code? Open in Web Editor NEWPython library for reading tiles from wsi tiff-files.
License: Apache License 2.0
Python library for reading tiles from wsi tiff-files.
License: Apache License 2.0
Opentile fails to open some ndpi files that lack non-essential properties.
Great work on opentile
๐
I wanted try it out just now and noticed you're using poetry caret version requirements to limit the major version of dependencies.
This causes issues for calendar versioned dependencies like tifffile and imagecodecs:
tifffile<2022.0.0,>=2021.6.14
imagecodecs<2022.0.0,>=2021.8.26
Where it's basically unpredictable anyways when a breaking change is introduced and currently the newest version is blocked off. It might also block off newer versions of dependencies that undergo a major version increase without breaking the part of the API that opentile uses.
Do you think it would make sense to just limit the minimum required version and run all tests in the CI regularly to detect and deal with incompatibilities with updated dependencies on a case by case basis?
If yes, I can work on a PR.
Cheers,
Andreas ๐
Just a heads up. This error was raised during testing of tifffile on Python 3.11:
============================================================================== FAILURES ==============================================================================
______________________________________________________________________ test_dependent_opentile _______________________________________________________________________
@pytest.mark.skipif(SKIP_PRIVATE or SKIP_LARGE or SKIP_WIN, reason=REASON)
def test_dependent_opentile():
"""Test opentile package."""
# https://github.com/imi-bigpicture/opentile
try:
from hashlib import md5
import turbojpeg
from opentile.geometry import Point
from opentile.ndpi_tiler import NdpiTiler
except ImportError:
pytest.skip('opentile missing')
fname = private_file('HamamatsuNDPI/CMU-1.ndpi')
turbo_path = os.path.join(
os.path.dirname(turbojpeg.__file__), 'turbojpeg.dll'
)
with NdpiTiler(
fname,
512,
turbo_path=turbo_path,
) as tiler:
# from example
> tile = tiler.get_tile(0, 0, 0, (0, 0))
tests\test_tifffile.py:16096:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
X:\Python311\Lib\site-packages\opentile\common.py:521: in get_tile
tiled_page = self.get_page(series, level, page)
X:\Python311\Lib\site-packages\opentile\ndpi_tiler.py:987: in get_page
ndpi_page = self._create_page(series, level, page)
X:\Python311\Lib\site-packages\opentile\ndpi_tiler.py:1078: in _create_page
return NdpiStripedPage(
X:\Python311\Lib\site-packages\opentile\ndpi_tiler.py:776: in __init__
super().__init__(page, fh, base_shape, tile_size, jpeg, frame_cache)
X:\Python311\Lib\site-packages\opentile\ndpi_tiler.py:459: in __init__
super().__init__(page, fh, jpeg)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <[AttributeError("'NdpiStripedPage' object has no attribute '_base_shape'") raised in repr()] NdpiStripedPage object at 0x20e988e0b10>
page = <tifffile.TiffPage 0 @184062235>, fh = <tifffile.FileHandle 'CMU-1.ndpi'>, jpeg = <opentile.jpeg.Jpeg object at 0x0000020E98927510>
def __init__(
self,
page: TiffPage,
fh: FileHandle,
jpeg: Jpeg
):
"""Ndpi page that should not be tiled (e.g. overview or label).
Image data is assumed to be jpeg.
Parameters
----------
page: TiffPage
TiffPage defining the page.
fh: FileHandle
Filehandler to read data from.
jpeg: Jpeg
Jpeg instance to use.
"""
super().__init__(page, fh)
if self.compression != 'COMPRESSION.JPEG':
> raise NotImplementedError(
f'{self.compression} is unsupported for ndpi '
'(Only jpeg is supported)'
)
E NotImplementedError: 7 is unsupported for ndpi (Only jpeg is supported)
X:\Python311\Lib\site-packages\opentile\ndpi_tiler.py:319: NotImplementedError
This is because of a change in the string representation of IntEnum members:
Python 3.11.0b3 (main, Jun 1 2022, 13:29:14) [MSC v.1932 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> from tifffile.tifffile import COMPRESSION
>>> str(COMPRESSION.JPEG)
'7'
Python 3.10.4 (tags/v3.10.4:9d38120, Mar 23 2022, 23:13:41) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from tifffile.tifffile import COMPRESSION
>>> str(COMPRESSION.JPEG)
'COMPRESSION.JPEG'
Hello everyone,
Is it possible to share the test images?
Or are they available somewhere online?
All the best,
Andreas
Pixel spacing and/or mpp is calculated wrong.
Striped ndpi-pages typically have stripe width of dividable by 256 (256, 1024, 4092) but other stripe widths can also be encountered, leading to incorrect tiling/crop.
If TurboJPEG should find the path to lib-turbojpeg library automatically, i.e. turbo_path is None, this fails as turbo_path is sent as str(turbo_path). Additionally, TurboJPEG_patch tries to use _find_turbojpeg() from TurboJPEG without extra __.
SvsTiler (and possibly also the PhilipsTiffTiler) incorrectly assumes that jpeg tables are stored separately from the frame data.
The example in the readme on how to read a tile using OpenTile is outdated.
Some svs-files have corrupt tiles at the edge of the image at levels other than the base level. Reading such tiles either produces zero-lenght byte or a technically valid tile but with mixed up content. Openslide fixes this by scaling tiles at corrupt edges from the base layer. A fix to do this in opentile is to be implemented.
Hello everyone,
The TurboJpegTest.test_multiple_extend
currently fails on Ubuntu and Windows because the test image height is just 512 and an OSError seems to be raised when cropping ranges are out of bounds.
Is a specific version of jpeg turbo required for the tests to pass?
Cheers,
Andreas ๐
________________________________________________________________________________________________ TurboJpegTest.test_crop_multiple_extend ________________________________________________________________________________________________
self = <tests.test_turbojpeg_patch.TurboJpegTest testMethod=test_crop_multiple_extend>
def test_crop_multiple_extend(self):
crop_parameters = [(0, 0, 1024, 1024)]
> crop = self.jpeg.crop_multiple(self.buffer, crop_parameters)[0]
tests/test_turbojpeg_patch.py:119:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../venv/lib/python3.9/site-packages/turbojpeg.py:671: in crop_multiple
self.__report_error(handle)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <opentile.turbojpeg_patch.TurboJPEG_patch object at 0x7f94e2ac4100>, handle = 44868608
def __report_error(self, handle):
"""reports error while error occurred"""
if self.__get_error_code is not None:
# using new error handling logic if possible
if self.__get_error_code(handle) == TJERR_WARNING:
warnings.warn(self.__get_error_string(handle))
return
# fatal error occurred
> raise IOError(self.__get_error_string(handle))
E OSError: Invalid crop request
../venv/lib/python3.9/site-packages/turbojpeg.py:883: OSError
======================================================================================================== short test summary info ========================================================================================================
FAILED tests/test_turbojpeg_patch.py::TurboJpegTest::test_crop_multiple_extend - OSError: Invalid crop request
libjpeg-turbo 3.0.0 has been released. This version seems to be compatible with opentile, i.e. lossless cropping is still possible. However the new version removes the comment marker and payload. Thus produced tiles have different checksum compared to when using older versions, and tests based on tile checksums will fail.
For ndpi pages that only contain one frame, cropping the frame with jpeg turbo also extends the frame to to the cropping region, e.g. cropping a 700x700 frame from position 512, 512 with size 512x512 gives a 512x512 frame. This however does not preserve the edges of the frame if the frame is not an even number of mcus in size or width. This result in a white border when viewing the level.
Simple solution is to use for example pillow to paste the frame into the extended frame. A (near) lossless solution is probably complicated as one needs to copy modified edge mcus into the frame.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.