Opening segmentation output about fast-pathology

[naguileraleal]

Sorry for not closing this issue before. I've not tried the groovy script again since all I needed was to import the .tiff segmentation in QuPath as an image. I did not need to generate QuPath annotation objects.
My original issue was solved

[smistad]

The segmentations output by FastPathology are stored as tiled image pyramids. However, since the model of "Epithelium segmentation in colonic mucosa (H&E Stained)" runs on a low magnification level, the output pyramid might just have one level, still it is tiled.

As far as I know, QuPath needs to convert segmentation masks into geometry. This groovy script: https://github.com/andreped/NoCodeSeg/blob/main/source/importPyramidalTIFF.groovy should allow you to import tiff segmentations for fastpathology into qupath. And this is the code that was used in the paper behind the "Epithelium segmentation in colonic mucosa (H&E Stained)" model.

[andreped]

Seems like you are trying to import the prediction as a regular image in QuPath. As @smistad mentions you will need to use the groovy script provided. The TIFF image is just an annotation image, but does not contain the original WSI. What you want to do is to import the annotation image as a QuPath annotation and connect it to the original WSI. This is only possible through a groovy script!

To get it working, try to do the following steps:

1. Add the relevant WSI in a QuPath project
2. Open the WSI of interest in QuPath
3. Add the groovy script in the QuPath script Editor
4. Modify the relevant parameters in the script (see here)
5. Run the groovy script

The result should then be stored as an annotation for the displayed WSI and hopefully be displayed. If not, either the script could not find the annotation image (see log), or the parameters were set wrongly, either not storing any annotations or drawing them incorrectly.

You can also run the groovy script on all the WSIs in the QuPath project, provided that you have run predictions on the same WSIs in a FP project.

@mhoibo and I just tried this script with the latest version of FP (v1.1.0) and it worked out of the box for a different application.

[naguileraleal]

Thanks for your quick response!

I was hoping that I could open the segmentation.tiff file as any other WSI, since I wanted to tile it and import it as masks in QuickAnnotator. Guess I won't be able to achieve that.

I knew that script from the NoCodeSeg project. I have already run it succesfully on other WSIs, although the generation of the .xml annotation file takes several hours (tested on Nuclei Segmentation, size of the WSI reported by QuPath: 86000x107000 pixels).

I tried running the script on the new segmentation anyways. The Groovy script produced this output:

ERROR: NullPointerException at line 101: Cannot invoke method getROI() on null object

ERROR: org.codehaus.groovy.runtime.NullObject.invokeMethod(NullObject.java:91)
    org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:44)
    org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
    org.codehaus.groovy.runtime.callsite.NullCallSite.call(NullCallSite.java:34)
    org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:47)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
    org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:130)
    Script18.run(Script18.groovy:102)
    org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:317)
    org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:155)
    qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:982)
    qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:914)
    qupath.lib.gui.scripting.DefaultScriptEditor.executeScript(DefaultScriptEditor.java:829)
    qupath.lib.gui.scripting.DefaultScriptEditor$ProjectTask.call(DefaultScriptEditor.java:1502)
    qupath.lib.gui.scripting.DefaultScriptEditor$ProjectTask.call(DefaultScriptEditor.java:1449)
    javafx.concurrent.Task$TaskCallable.call(Task.java:1426)
    java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    java.base/java.lang.Thread.run(Unknown Source)

I tried various values of the parameters "downsample" (0, 1, 2, 10) and "level" (0, 1) of the script, without success. I'll include the values of the parameters now.

// --- SET THESE PARAMETERS ---
def masksPath = 'C:\\Users\\nicol\\fastpathology\\projects\\glandulas\\results'  // path to where TIFFs are stored (FastPathology project)
def downsample = 0                      // which scaling factor to use, if the segmentation is produced at a lower magnification level
def level = 0                           // which level to extract segmentation from (choosing 0 may be slow)
def extension = ".tiff"                 // pyramidal TIFF
def classNames = ["Background", "Epithelium"]   // names of classes of interest (in this case two classes + background)
def taskName = "Epithelium segmentation in colonic mucosa (HE-Stained)"    // name of task, which correspond to which task that was run in FastPathology to get this result
int channel = 0                         // 0-based index for the channel to threshold
def fromFP = true                       // whether result is from FastPathology or not, if no, we assume that the TIFF lie directly in the maskPath directory with the same name.
// ----------------------------

This is the pipeline.fpl that generated the segmentation

PipelineName "Epithelium segmentation in colonic mucosa (HE-stained)"

PipelineDescription "Segmentation of epithelial cells in HE-stained biopsies of colonic mucosa of active and inactivate inflammatory bowel disease. For more info see the article 'Code-Free Development and Deployment of Deep Segmentation Models for Digital Pathology' Frontiers in Medicine 2022 https://www.frontiersin.org/articles/10.3389/fmed.2021.816281/full and also the webpage https://github.com/andreped/NoCodeSeg."

PipelineInputData WSI "Whole-slide image"

PipelineOutputData segmentation stitcher 0

Attribute classes "Background;Epithelium"



### Processing chain

ProcessObject tissueSeg TissueSegmentation

Attribute threshold 70

Input 0 WSI



ProcessObject patch PatchGenerator

Attribute patch-size 512 512

Attribute patch-magnification 10

Attribute patch-overlap 0.05

Attribute mask-threshold 0.02

Input 0 WSI

Input 1 tissueSeg 0



ProcessObject network SegmentationNetwork

Attribute scale-factor 1.0

Attribute model "$CURRENT_PATH$/../colon-epithelium-segmentation-he-model/HE_IBDColEpi_512_2class_140222.onnx"

Input 0 patch 0



ProcessObject stitcher PatchStitcher

Input 0 network 0



### Renderers

Renderer imgRenderer ImagePyramidRenderer

Input 0 WSI



Renderer segRenderer SegmentationRenderer

Attribute opacity 0.5

Attribute border-opacity 1.0

Input 0 stitcher 0

Here's the link to the segmentation generated https://drive.google.com/file/d/1-7b6XHk4aPnzyVKxTTqKwa4oVKVXQ3TO/view?usp=share_link (couldn't upload it to github). I'll ask for permission for uploading the WSI associated with the segmentation, and I'll share it with you if I can.

[andreped]

Try removing "Background" in the class names of the gropvy script. There is only one class of interest. Hence, there should only be one string in the list, i.e., Epithelium.

I see now that the comment I made was confusing. I will correct that. Cheers!

Just remember to set the appropriate downsample. For instance, if your WSI was 40x and you ran inference on 10x, you should set downsample=4.

[andreped]

I was hoping that I could open the segmentation.tiff file as any other WSI, since I wanted to tile it and import it as masks in QuickAnnotator. Guess I won't be able to achieve that.

Oh, but yes you can, @naguileraleal. The reason why the image is all "black" is because the pyramidal TIFF annotation image is stored with uints 0 and 1 (if you are doing binary segmentation), and the renderer in QuPath expects dynamic range to be in range [0, 255] for uints, hence, it interprets uint 1 as almost black. As the difference is so small, there is hardly any contrast, and thus very challenging to see the annotations. I have observed the same when saving PNGs in a similar manner. I would think you might have the same issue when attempting to view the generated tiles on disk.

As a sanity check, to view the annotations, you could perform simple thresholding in QuPath (see here). Try setting the threshold to 0 or 1.

Hence, regardless of thresholding, you should still be able to extract tiles in QuPath and convert annotations to XML, as you intended to do.

For exporting tiles, be sure to set the parameters to appropriate values, as you now are using the annotation image, and then glass detection with threshold 50 does not really make sense. Try setting it to 255 (see here). For more information on this threshold, see usage here. Might be that this script expects there to be an annotations. In your case your WSI is the actual annotation. Hence, what you want to do is just extract tiles from the "WSI/annotation" image. If you observe any issues, this example from the documentations might be helpful.

[naguileraleal]

Hi! Sorry for the delay.

Try removing "Background" in the class names of the gropvy script. There is only one class of interest. Hence, there should only be one string in the list, i.e., Epithelium.

After removing the "Background" class from the list, the script ran correctly, and I was able to import the annotations to QuPath in XML format.

As a sanity check, to view the annotations, you could perform simple thresholding in QuPath (see here). Try setting the threshold to 0 or 1.

Your suggestions for correctly visualizing the output segmentation from FastPath in QuPath worked beautifully. I was then able to cut the thresholded .tiff into tiles. Thanks! I was only able to only do this for the output of the "Epithelium segmentation in colonic mucosa (H&E Stained)" model, though. When trying with the output of the "Nuclei Segmentation" model, I was not able to load the segmentation.tiff output into QuPath. The error message is "Added images (1/1) - image null" and then the image never appears in the list of images of the QuPath project. This file is considerably larger than the previous one (300MB vs 7MB).

[andreped]

My thresholding "trick" was only relevant to check that there were annotations in the image and not just black.

After importing, you could try to export the image as tiles. However, might be that you need to convert the image to proper annotations first, for instance by thresholding to generate annotation objects.

The error message for the nuclei segmentation does not tell me much tbh.

---

EDIT: @naguileraleal I have been able to import nuclei segmentations in QuPath before. However, it might be that you are running into memory issues. Could you try monitoring the CPU memory usage while you are importing?

Also, as nothing is shown, I would assume that there are no annotations in the annotation bar after the groovy script is done?

Another reason why nothing is shown could be that the parameters in the groovy script are set wrongly. For nuclei segmentation, I would set something like this:

// --- SET THESE PARAMETERS ---
def masksPath = 'C:\\Users\\nicol\\fastpathology\\projects\\glandulas\\results'  // path to where TIFFs are stored (FastPathology project)
def downsample = 4                      // which scaling factor to use, if the segmentation is produced at a lower magnification level
def level = 0                           // which level to extract segmentation from (choosing 0 may be slow)
def extension = ".tiff"                 // pyramidal TIFF
def classNames = ["Nuclei"]   // names of classes of interest (in this case two classes + background)
def taskName = "Nuclei segmentation"    // name of task, which correspond to which task that was run in FastPathology to get this result
int channel = 0                         // 0-based index for the channel to threshold
def fromFP = true                       // whether result is from FastPathology or not, if no, we assume that the TIFF lie directly in the maskPath directory with the same name.
// ----------------------------

The value downsample = 4 assumes that the model was run on a 4x poorer resolution, e.g., WSI is 40x but model was run at 10x.

[naguileraleal]

Running the script on this project results in the following error

WARN: Unable to open file:/C:/Users/nicol/fastpathology/projects/nucleos/results/44-D5/Nuclei%20segmentation/Segmentation/Segmentation.tiff with OpenSlide: TIFFRGBAImageGet failed

Memory values are normal (~7GB out of 15GB).

Anyways, the output of this script is not so relevant to my project as it is being able to load the .tiff file in QuPath.

Here is the WSI on which I ran the Nuclei Segmentation model in FastPath. This in turn generated the Segmentation.tiff file which I'm now trying to import into QuPath (without success). I'll leave it here just in case you want to give it a try.

[andreped]

@naguileraleal Did you manage to resolve this issue? Seems like others are not having this issue, at least not with the latest artifact(s) (e.g., see here).

[andreped]

Sorry for not closing this issue before.

No worries! :] Feel free to open a new issue or discussion, if you are experiencing any others issues/obstacles. We have more time dedicated this fall to fix issues and improve FP.