Rendering offset for some WSI formats about fast-pathology

[smistad]

Hello @andreped @cavenel and @vladpopovici

Sorry for responding so slowly to this.

I have now found the reason for this error.
It was due to a rounding error of the pixel spacing in FAST.
After this fix, the segmentation renders at the correct location.

I will commit a fix to FAST soon, then we will need to rebuild FAST-Pathology with the new fix.
Stay tuned.

[cavenel]

I can confirm that it looks amazing now!

Thank you!

[smistad]

@cavenel I just tested the nuclei segmentation model on the data you sent @andreped with the fix, and it renders correctly now.

[smistad]

@cavenel I have now rebuilt FAST-Pathology with the fix.

You can download installers from artifacts here:
Ubuntu: https://github.com/AICAN-Research/FAST-Pathology/actions/runs/5809496671
Windows: https://github.com/AICAN-Research/FAST-Pathology/actions/runs/5809495694
Mac: https://github.com/AICAN-Research/FAST-Pathology/actions/runs/5809497676

Let us know how it goes

[vladpopovici]

I can confirm as well that the patch fixed the issue. Thanks a lot!

[vladpopovici]

Hello,

I have observed the same behavior on .MRXS whole slide images, on Windows, with latest release (v1.1.0), with all segmentation models. I can provide a test image if needed.
Thanks,
Vlad

[vladpopovici]

I can confirm the observations made by @pr4deepr:

-original .MRXS from 3DHistech scanner is read by FP, but segmentation is shifted;
-.MRXS -> .TIFF converted by 3DHistech software (SlideMaster) results in a file that cannot be read in FP;
-.MRXS -> .MRXS results in a file that can be read in FP, but the segmentation is shifted;

Additionally, I tried .MRXS -> OME-TIFF (with bioformats tools: bioformats2raw + raw2ometiff which results in a file that cannot be read by FP ("level is too large to convert into a FAST image").

[vladpopovici]

The bfconvert produces a corrupted image that vips fails to convert afterwards.
But I have used this command, which should be (?) equivalent:

> vips openslideload <file>.mrxs <result_file>.tiff[tile,tile-width=1024,tile-height=1024,compression=jpeg,Q=100,bigtiff,pyramid] --vips-progress

The resulting file is perfectly opened by FP, but the detection mask is, again, shifted. I will have to check in the code to see which meta-parameters are causing this behavior. It's really frustrating... :)

[andreped]

Just observed the same using the latest artefact when testing a custom mitosis segmentation model.

I was running a lot of different tests. I could not reproduce this issue on the CellSens VSIs, but for the generic TIFF images, it seems to reproduce quite consistently.

However, I find it strange, as these images were originally stored as .vsi, but converted to .tiff. I believe this used to work before (in the first release of FP). Not sure what changed. Might also be that the converter stores some spacing data wrong.

There should be some WSIs located on the SINTEF-server you could test:
/mnt/EncryptedPathology/pathology/images_tiff_tz_1024_jpeg_85/T.tif

I observed the same issue using the nuclei segmentation method that is part of the software. If that WSI is not in the path, you could test any of the .tif-files. I believe it should be consistent among those files, but not sure.

[smistad]

This is caused by pixel spacing of segmentation being calculating incorrectly.

We are not able to extract spacing information from the VSI format, that is why it is not occurring on these images.

[andreped]

We are not able to extract spacing information from the VSI format

I'm observing this issue on the TIFF images, not the VSI images. But the TIFF images were originally in the VSI format. The TIFF images are located at:
/mnt/EncryptedPathology/pathology/images_tiff_tz_1024_jpeg_85/

The corresponding VSI images are located at:
/mnt/EncryptedPathology/pathology/images/

on the SINTEF server, if I recall correctly, if you wish to check this further.

[andreped]

That's interesting, @vladpopovici! Thanks for reporting it!

@smistad is on vacation now and I am preoccupied with some other deadlines, so we will not be able to look into this until August earliest. Will keep you updated!

Regarding test data, could you reproduce the issue on any of the MXRS files that are part of the OpenSlide test data (see here).

[vladpopovici]

Now that's awkward: on the test data the segmentation mask is spot on! I will take a closer look to see what are the differences in metadata between test images and mine. My images are produced by the scanner software for 3DHistech. Keep you posted. V.

[andreped]

My images are produced by the scanner software for 3DHistech.

Oh, yeah. This we have seen in a separate issue. See issue #13 (or more specifically, see this comment). Basically, the MRXS format produced from the 3DHistech format has been modified, which results in OpenSlide failing to interpret it correctly. This was also talked about thoroughly in this thread.

To get these images to work with non 3DHistech-products, you will need to use the 3D Histech converter tool (see here). Which should resolve this issue, but I have myself never tried it. Could you give it a go to see if it resolves the issue you are experiencing on your local WSI?

[andreped]

The annoying problem is basically that the 3DHistech have corrupted the original MRXS format slightly, meaning that for FP the regular OpenSlide test MRXS files reads as expected, but the one directly for the 3DHistech server does not.

-.MRXS -> .TIFF converted by 3DHistech software (SlideMaster) results in a file that cannot be read in FP;

I believe that is because it does not produce a pyramidal TIFF, which is needed to be read by FP. If you try to open it in QuPath, it will likely open, it might take a while and prompt a warning, as it will try to pyramidize the entire image before reading it.

Additionally, I tried .MRXS -> OME-TIFF (with bioformats tools: bioformats2raw + raw2ometiff which results in a file that cannot be read by FP ("level is too large to convert into a FAST image").

That is related to the previous comment. I would recommend using bioformats + vips instead. That is you first convert the WSI to BigTIFF and then to a pyramidal TIFF. See this example python script on how to do it. Note that I just use two command line tools. Below is a simple example of how to do it:

# original format -> btf
> sh /path/to/bfconvert -tilex 1024 -tiley 1024 -nogroup -no-upgrade -overwrite -bigtiff -series 0 /path/to/original/image.format /path/to/temporary/image.btf

# btf -> pyramidal tif
> vips tiffsave /path/to/temporary/image.btf /path/to/converted/image.tif --bigtiff --tile -pyramid --compression=jpeg --Q=85

See the aforementioned script for more information regarding compression in the first step and bfconvert path.

[andreped]

I believe by attempting to convert in another software than 3DHistech's own converter, you will likely still get the offset. Something in the metadata of the raw MRXS format you are getting from the 3DHistech scanner is likely not following the MRXS standard. They likely added this annoyance to force people to use their products. I have seen the same with Philips scanners before.

Did you find a good workaround for this issue, @pr4deepr?

Did you try to use the converter I mentioned previously. Might be that it has been rebranded to slidemaster.

[vladpopovici]

Yes, I tried with SlideMaster - see my attempts above.
Anyhow, I did not see this kind of artifact in other programs analyzing the same images (including QuPath, HALO and my own scripts :) ). I used OpenSlide previously and I was able to correctly align the masks and navigate the images.
I will try digging into the code of FAST library.

[andreped]

I guess this is something that could be addressed in FAST then. We have a similar issue with some other WSI formats, so it should be possible to resolve. However, @smistad is on vacation, so it will have to wait until August.

[cavenel]

Hi,

As discussed on https://forum.image.sc/t/nocodeseg-deep-learning-based-segmentation-without-code-qupath-deepmib-fastpathology/60069/10, I have the exact same shifting problem in FAST (via FastPathology).
The WSI file is from 3DHistech, and I attach the openslide properties here: openslide_output.txt

And the shift looks like this:

I can send a test image if needed.
This is a bit blocking for me, as no conversion worked: vips keeps the shift issue, and QuPath makes an OME.TIFF that FAST can not open.

[andreped]

I can send a test image if needed.

Hello, @cavenel! It would be great if you could share an image, such that we could see if we could reproduce the issue. It would also help us in debugging the issue and potentially finding a fix.

If you do not wish to make it open to everyone, you could share the WSI with me at: [email protected]

[andreped]

Note that we have resolved similar issues before for a difference image format: #61 (comment)

Also note that for this exact format, @pr4deep seemed to have managed to resolve this issue for the same 3DHistech MRXS format by converting the format to TIFF using 3DHistech's own converter: #13 (comment)

@cavenel Could you please check again that you did the conversion correctly. This is also relevant for @vladpopovici (see comment below).

-.MRXS -> .TIFF converted by 3DHistech software (SlideMaster) results in a file that cannot be read in FP;

@vladpopovici I believe the conversion you did here actually fixed it, but you have to convert the image to pyramidal TIFF, not regular TIFF, as FastPathology only works with pyramidal images, in contrast to QuPath that can pyramize the image if necessary.