Reading WSI with .tif and ome.tiff format about pathml

[jacob-rosenthal]

Hi, from the error message it looks like it may be a problem with the format of the image which is causing opencv to be unable to process it.
Can you check the shape (i.e. number of channels) and data type of the images you are trying to use?

Labels need to be added manually

[maryamahz]

Thanks for your response.
I tried to read the sample image on Pathml (CMU-1.tif). But I get the same error.

from pathml.core import SlideData, SlideType
from pathml.preprocessing import Pipeline, BoxBlur

# load the image
he_type = SlideType(stain="HE", platform=None, tma=False, rgb=True, volumetric=False, time_series=False)
wsi = SlideData(".../CMU-1.tif", slide_type = he_type)
print('wsi', wsi)
pipeline = Pipeline([
    BoxBlur(kernel_size=15)
])

from dask.distributed import Client, LocalCluster

cluster = LocalCluster(n_workers=6)
client = Client(cluster)
wsi.run(pipeline, distributed=True, client=client);

Here is the error:

wsi SlideData(name='CMU-1.tif',
	slide_type=SlideType(stain=HE, platform=None, tma=None, rgb=True, volumetric=None, time_series=None),
	filepath='.../CMU-1.tif',
	backend='bioformats',
	image shape: (27712, 40000),
	number of levels: 7,
	0 tiles: [],
	0 masks: [],
	labels=None,
	counts=None)

[maryamahz]

Update: Since the format of the “CMU-1.tif” image is .tif and Openslide supports this format. I intentionally changed the backend from "bioformats” to "openslide" and did not get the above error.

However, in order to read the image with ome.tiff format, I need to use the backend = "bioformats". Which gives me the same error. It would be great if you could help me with this issue.

[sreekarreddydfci]

Hi,
Looks like the issue is with the file format. Can you share sample files, to replicate the error? I was able to load tiff and ome-tif files without any issue.

[jacob-rosenthal]

@maryamahz what are the version of opencv, javabridge, and bioformats that you have installed in your environment?

Are you able to load any images successfully using the bioformats backend? Can you please try with small_vectra.qptiff?

[maryamahz]

Thanks for getting back to me. Here are the versions of packages that are installed (I used Docker image to install PathML):
opencv==4.7.0.
bioformats==4.0.0
javabridge==4.0.0

Here is the image I am using as a sample: https://openslide.cs.cmu.edu/download/openslide-testdata/Generic-TIFF/

Could you also please share your codes and sample image, so I can replicate the process?

Yes, As you can see above, I can load/read the images successfully using the Bioformats backend. But when I apply a filter using this command:

wsi.run(pipeline, distributed=True, client=client);

I get the above error.

I also tried on small_vectra.qptiff. But got the same error :(

[sreekarreddydfci]

Here's the code

from pathml.core import SlideData, SlideType
from pathml.preprocessing import Pipeline, BoxBlur

# load the image
he_type = SlideType(stain="HE", platform=None, tma=False, rgb=True, volumetric=False, time_series=False)
wsi = SlideData("../CMU-1.tiff", slide_type = he_type, backend= 'openslide')
print('wsi', wsi)
pipeline = Pipeline([
    BoxBlur(kernel_size=15)
])
wsi.plot()

from dask.distributed import Client, LocalCluster

cluster = LocalCluster(n_workers=6)
client = Client(cluster)
wsi.run(pipeline, distributed=True, client=client);

Output

SlideData(name='CMU-1.tiff',
	slide_type=SlideType(stain=HE, platform=None, tma=None, rgb=True, volumetric=None, time_series=None),
	filepath='../CMU-1.tiff',
	backend='openslide',
	image shape: (32914, 46000),
	number of levels: 9,
	22912 tiles: ['(0, 0)', '(0, 1024)', '(0, 10240)', '(0, 10496)', '(0, 10752)', '(0, 11008)', ...],
	0 masks: [],
	tile_shape=(256, 256, 3),
	labels=None,
	counts=None)

This worked with small_vectra.qptiff as well. Could you try restarting the kernel?

[maryamahz]

Thank you for sharing the code.
Yes, this works well for me. As I mentioned before, the reason is that here you are using backend='openslide'.
I believe when the backend is 'openslide', it works well. But I get the above-mentioned error when I want to read/process an image with ome.tiff format that needs to be read by backend = 'bioformats' or when I want to read tiff format with 'bioformats'.

[jacob-rosenthal]

@maryamahz Ultimately this error message you are seeing is coming from openCV, so it is likely due to an incompatibility between your input image and the opencv library. These errors are usually caused when using images with shapes or datatypes not supported by that particular opencv function. So we should be able to reproduce the error using only numpy and opencv, since the error isn't arising from pathml itself.

I tried playing around to see if I could reproduce the error, and I found that it comes up when trying to use box blur on an image with an empty dimension. It works fine for images of shape (n, n, 1), (n, n, 2), (n, n, 3), (n, n, 4), etc... but fails and throws the same error message on an image with shape (n, n, 0). I checked and this is the case for both integer images and float images.

>>> import numpy as np
>>> import cv2
>>> im = np.random.randint(low=0, high=250, dtype=np.uint8, size=(256, 256, 0))
>>> print(f"{im.shape=}")

im.shape=(256, 256, 0)

>>> im_blurred = cv2.boxFilter(im, ksize=(15, 15), ddepth=-1)

Traceback (most recent call last): File "<stdin>", line 1, in <module> cv2.error: OpenCV(4.8.0) /Users/xperience/GHA-OpenCV-Python/_work/opencv-python/opencv-python/opencv/modules/core/src/matrix.cpp:1099: error: (-215:Assertion failed) dims <= 2 && step[0] > 0 in function 'locateROI'

Can you verify the dimensions of the image you are working with?

You can try loading the image, extracting a single tile, looking at the image array contained in the Tile object, and seeing what the shape and datatype of it are. That information would help us get to the bottom of this problem.
Additionally, if you could do the same for both openslide and bioformats backend, we could hopefully find what the difference is in the image array which is causing this problem.

[maryamahz]

While reading the .tiff format image with ‘bioformats’ the image shape is (256, 256, 1, 3, 1). But with ‘openslide’ the shape is (256, 256, 3). The type for both is unit8.

[jacob-rosenthal]

Ah ok that makes sense!
Yes, the expected behavior for bioformats backend is to return images in (X, Y, Z, C, T) dimensions. But, opencv probably wants your images to have only 3 dimensions (X, Y, C).

To solve, try adding CollapseRunsVectra() transform as the first step of your pipeline. This transform basically just applies np.squeeze to collapse the dimensions of size 1

Let me know if this works!

[maryamahz]

Yes! the error is fixed.
Thanks

[jacob-rosenthal]

Great! Closing this now as resolved, feel free to reopen if it comes up again.

[xiachenrui]

Thanks for your discussion. It is also useful for me.