aican-research / fast-pathology

⚡ Open-source software for deep learning-based digital pathology

FastPathology

Open-source software for deep learning-based digital pathology

FastPathology was created by researchers at SINTEF and the Norwegian University of Science and Technology (NTNU). A paper presenting the software and some benchmarks has been published in IEEE Access.

💻 Getting started

To install FastPathology, follow the instructions for your operating system:

Windows (10 or newer)

• Download and run the Windows installer from the release page. Note: Windows might prompt you with a security warning, to proceed you must press "More info" followed by "Run anyway".
• Run fastpathology from your start menu.
• To uninstall the application, go to start menu -> remove programs -> find fastpathology and select uninstall. Optionally you can also delete your C:/Users/"your username"/fastpathology/ which includes stored project results, pipelines, and models. And the folder C:/ProgramData/FAST/ which contains a cache.

Ubuntu Linux (18.04 or newer)

• Install OpenCL for Linux by downloading an implementation depending on the CPU/GPU you have:
  • NVIDIA - Install CUDA.
  • Intel - Install the OpenCL NEO driver.
  • AMD - Install the ROCm stack.
  • If none of the above fits, you can try the Portable Computing Lanauge (PCOL), although reduced performance is likely.

• Download the debian package from the release page.
• Install the debian package from the terminal or by double-clicking it:

sudo dpkg -i fastpathology_ubuntu*.deb

• Go to the folder /opt/fastpathology/bin/ and run the fastpathology executable, or run it from the ubuntu menu (windows button->type fastpathology).
• To uninstall the application, run the following in your terminal:

sudo apt remove fastpathology
# Optionally, you can also delete your fastpathology folder 
# which includes stored project results, pipelines and models.
# and the FAST folder which stores cache files.
rm -Rf $HOME/fastpathology
rm -Rf $HOME/FAST

macOS (10.13 or newer)

Note that the macOS version of FastPathology is experimental.

• Install homebrew if you don't already have it. Then, install the following packages using homebrew:

brew install openslide libomp

• Download and run the macOS installer (.dmg) from the release page. Download the x86_64 file if you have an Intel CPU, or download the arm64 file if you have an Apple Silicon (M1/M2) CPU.
• In the installer, drag-and-drop the FastPathology bundle to the Applications directory.
• When the copy is finished, double-click the Applications icon in the installer and right-click + open FastPathology from the Applications menu.
• A warning should be prompted "macOS cannot verify the developer of FastPathology (...)", click open and the program should launch.

Note: This is only required to be done once. For all future usage, launch FastPathology as a regular App bundle.

• To uninstall the application, delete the extracted folder. Optionally, you can also delete the /Users/"your username"/fastpathology/ folder, which includes stored project results, pipelines, and models. And the folder /Users/"your username"/FAST/ which contains a cache.

Optional: NVIDIA GPU Inference

If you have an NVIDIA GPU on your machine you can enable high-speed inference by downloading and installing the following:

• CUDA 11
• cuDNN 8.2
• TensorRT 8.2

Note: Make sure to download the correct versions. NVIDIA GPU inference is not supported on Mac.

License

The source code of FastPathology is licensed under the BSD 2-clause license, however the FastPathology program use and are linked with many great third-party libraries which have several different open source licenses, see the licenses folder in the installation folder for more details.

📹 Demos and tutorials

Very simple demonstrations of the platform can be found on Youtube. More in-depth demonstrations will be added in the future. Wikis and tutorials can be found in the wiki. More information can be found from the pages section on the right in the wiki home.

🎊 Features

The software is implemented in C++ based using FAST. A wide range of features have been added to make working with whole slide images (WSIs) a piece of cake!

• Graphical User Interface - User-friendly GUI for working with WSIs without any code interaction.
• Deep learning - Deployment and support for multi-input/output convolutional neural networks (CNNs).
• Visualization - Real-time streaming of predictions on top of the WSI with low memory cost.
• Use cases - Patch-wise classification, low and high-resolution segmentation, and object detection are supported.
• Inference Engines - FAST includes a variety of different inference engines, i.e. TensorFlow CPU/CUDA (support both TF v1 and v2 models), TensorRT (UFF and ONNX), OpenVINO (CPU/GPU/VPU), and ONNX Runtime (CPU/GPU).
• Text pipelines - Possibility to create your own pipelines using the built-in script editor.
• Formats - Through OpenSlide FastPathology supports various WSI formats.

🔬 Applications of FastPathology

• Pettersen et al., Code-free development and deployment of deep segmentation models for digital pathology (2022), Frontiers in Medicine, https://doi.org/10.3389/fmed.2021.816281
• Pedersen et al., H2G-Net: A multi-resolution refinement approach for segmentation of breast cancer region in gigapixel histopathological images (2022), Frontiers in Medicine, https://doi.org/10.3389/fmed.2022.971873
• Røyset et al., Deep learning-based image analysis reveals significant differences in the number and distribution of mucosal CD3 and γδ T cells between Crohn's disease and ulcerative colitis (2022), The Journal of Pathology, https://doi.org/10.1002/cjp2.301

🔨 Development setup

1. Either
   • Download and install a release of FAST.
   • Compile and install FAST on your system.
2. Clone this repository

   git clone https://github.com/AICAN-Research/FAST-Pathology.git

3. Setup build environment using CMake
   Linux (Ubuntu)

   mkdir build
   cd build
   cmake .. -DFAST_DIR=/path/to/FAST/cmake/

   Windows (Visual Studio) Modify generator -G string to match your Visual studio version. This command will create a visual studio solution in your build folder.

   mkdir build
   cd build
   cmake .. -DFAST_DIR=C:\path\to\FAST\cmake\ -G "Visual Studio 16 2019" -A x64

4. Build

   cmake --build . --config Release --target fastpathology

5. Run Linux (Ubuntu)

   ./fastpathology

   Windows

   cd Release
   fastpathology.exe

NOTE: Visual Studio 19 has been tested with both FAST and FastPathology and works well.

✨ How to cite

Please, consider citing our paper, if you find the work useful:

  @article{9399433,
  author={Pedersen, André and Valla, Marit and Bofin, Anna M. and De Frutos, Javier Pérez and Reinertsen, Ingerid and Smistad, Erik},
  journal={IEEE Access}, 
  title={FastPathology: An Open-Source Platform for Deep Learning-Based Research and Decision Support in Digital Pathology}, 
  year={2021},
  volume={9},
  number={},
  pages={58216-58229},
  doi={10.1109/ACCESS.2021.3072231}}

fast-pathology's Issues

macOS installer

As there are some users on macOS, it would be beneficial to have an installer. This would:

• Make the installation behaviour similar across all operating systems
• Ability to run the software similarly as for other softwares - FP will be available as an APP in macOS

Ubuntu version not working

Tried downloading the latest artifact on Ubuntu 18.04. Not working.

Did sudo dpkg -i fastpathology_ubuntu18.04_1.0.0.deb and run it. Nothing happened. Checked /opt/fastpathology/, it is there. Tried running the program from the bin/ directory, and this prompted:

INFO [139839184267968] Found variable filename in pipeline file
ERROR [139839184267968] Terminated with unhandled exception: The pipeline file requires you to give a value for the variable named filename
This is done by adding --filename <value> to the command line arguments
Aborted (core dumped)

Old release works just fine.

Improve feedback during inference

When running the model, a progress bar shows up, but there is little to no real feedback to the user regarding how far it has come (or whether it is processing at all).

The only real feedback is when running a pipeline on all images in a project.

We should improve the progress widget by perhaps add a logger that prompts that it is indeed working, and perhaps highlight which step it is currently working on, if possible/relevant.

Some patches are not rendered at different image planes

When running the latest release on Win10, I observed that there were some holes in the segmentation mask, when running the nuclei segmentation model.

It might be that either:

1. something went wrong during stitching
   or
2. some patches are skipped for unknown reasons when rendering.

Progressbar works differently than expected in batch mode

I just tried running inference in batch mode using the two images in the test data using the BACH model.

What happens is that a progress bar shows. All good. However, the progressbar seems to work a little differently than assumed.

What happens is that it iterates up to 50%, patch-by-patch, before it goes to the next image, and continues to 100%.

At least the user has verbose for both images, but there is no information on how many images it has iterated through, unless you do the math yourself.

I would think two progressbars here make more sense and is more user friendly:

1. showing which image it is currently processing
2. for the current image, how far has the patch processing come

Rendering offset for some WSI formats

Just run the nuclei segmentation pipeline which follows with FP on a local WSI, and observed that the rendered segmentations were off.

This was observed for a WSI stored in the generic TIFF format, on Windows 10, using the latest artifact.

See example below.

OpenGL Window flickers

When running nuclei segmentation, I observed that the GUI was flickering.

See no reason why this should happen, as predictions are not being streamed to the renderer.

Tested on Windows 11, using Dell XPS 13, on CPU.

cmake .. -D FAST_DIR=D:\FAST\cmake\ -G "Visual Studio 17 2022" -A x64

Hello, I'm having problems in the compilation process.

D:\FAST-Pathology\build>cmake .. -D FAST_DIR=D:\FAST\cmake\ -G "Visual Studio 17 2022" -A x64
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19044.
CMake Error at CMakeLists.txt:13 (find_package):
By not providing "FindFAST.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "FAST", but
CMake did not find one.

Could not find a package configuration file provided by "FAST" with any of
the following names:

FASTConfig.cmake
fast-config.cmake

Add the installation prefix of "FAST" to CMAKE_PREFIX_PATH or set
"FAST_DIR" to a directory containing one of the above files. If "FAST"
provides a separate development package or SDK, be sure it has been
installed.

-- Configuring incomplete, errors occurred!
See also "D:/FAST-Pathology/build/CMakeFiles/CMakeOutput.log".

How can I solve this problem? I look forward to your reply. Thank you.

Optimize font sizes

The current font sizes in the GUI is quite large, and inconsistent.

Might be that this font size works better for very large screens, or with higher resolution.

Could be a good idea to either:

1. dynamically adjust font size based on screen resolution at program initialization
2. be able to set the font size yourself through a settings widget.

Advanced mode does nothing

This is an old feature, which we most likely should just remove.

Perhaps, we could add a developer option, in a settings widget in the future, but for the stable release, I would just remove it.

Updating the import pyramidal tif script to the new folder structure

I think this script needs to be updated to work with the new folder structure for results from FastPathology:

https://github.com/andreped/NoCodeSeg/blob/main/source/importPyramidalTIFF.groovy

The script assumes a different folder structure than what was before and also assumes the .tiff file has the same name as the image, whereas the newest update produces images named "segmentation.tiff".

cmake --build . --config Release --target fastpathology

Hello, thank you very much for sharing the code.
I got stuck in that build step when compile the fastpathology.
I have the following problems.
This bug has been bothering me for several days. I have encountered this problem in ubuntu and windows.

“无法解析的外部符号“ means an external symbol that cannot be resolved in the picture.

Looking forward to your reply.

Some issues with display of rendering, etc.

I have briefly tested the new FastPathology version. More stable. but some issues:

1. crashes when I try to load small tiff files, but not large ones. When I test prediction for mitoses they become invisible at "overview" ca. 4x magnification and thus it becomes almost pointless to run the algorithm when you have to look for all green dots. In Qupath, small annotations are magnified when you zoom out so they are still visible. Is this possible to get into FastPathology (that all annotations have a given line thickness regardless of magnification?)

2. The NoCodeSeg import script does not work on the tiff file I got out in the results folder. Correct size, but too large for the import script (this was a standard WSI, but large about 60k x 100k pixels, and MIB/DeepMIB/Matlab struggles with these as well at 40x resolution).

3. The import script for direct import from fast pathology at the NoCodeSeg page probably does not work either as it is a new folder structure with lots of subfolders. I would have dropped all these subfolders in results, or at least had a little less of them.

4. It is difficult to know whether the algorithm is actually running or not - no progress bar and no time indication for estimated finished prediction. For mitoses, I waited 15 minutes and it was finished, but could just as easily have been nothing that happened - I did not know that as a user.

Improve layer rendering

When rendering results, for instance for nuceli, the segmentations become quite invisible when zooming out.

I think it might be a good idea to add some type of rendering effect which enables it to visualize segmentations also when zoomed out. For instance if there are lots of nuclei at the same spot, all colored green, if you zoom out, all segmentations should merge into a "larger segmentation", which can be visualized.

Perhaps this is something we could enable/disable, but I think that might look better.

What are your thoughts?

Removing WSI from project not working properly

Just remembered that @dbouget added a feature to remove WSIs from the project, by right-clicking the thumbnail image.

However, if you do that, the QStackedWidget is not properly updated, which leaves an empty item in it.

I believe this can be fixed by force updating the widget through QApplication::processEvents() or similar.

unable to import savedModel (tf) format

Just tried to import model trough the gui, the pipeline works fine, but was not able to upload the model as a folder.

Smarter rendering of small objects

Similar to #32, if you zoom out, small objects become invisible. In that scenario it makes sense to merge all together, or similar, if they are close, when rendering are low resolution.

However, for mitoses, which commonly are quite rare in a WSI, you will not have the same effect. Perhaps it could be a good idea to support drawing disks (or similar) of different sizes, such that when you zoom out, you can visualize where the mitoses are located in a WSI.

That would be a great solution for visualizing how the mitoses are distributed in a WSI, and it enables the user to very easily find the hotspot, relevant for measuring the mitosis activity.

Add license

Use same license as FAST?

macOS version not working

I just tried downloading the recent artifact and running the program inside the bin/ directory, this resulted in:

dyld[22435]: Library not loaded: @rpath/libFAST.4.dylib
  Referenced from: /Users/sintefmedtek/Downloads/fastpathology_macos10.13_1.0.0/bin/fastpathology
  Reason: tried: '$ORIGIN/../lib/libFAST.4.dylib' (no such file), '$ORIGIN/../lib/libFAST.4.dylib' (no such file), '/usr/local/lib/libFAST.4.dylib' (no such file), '/usr/lib/libFAST.4.dylib' (no such file)

All dependencies should lie in the lib/ directory, and they are, but seems like FP is unable to find them. Perhaps something is wrong with @rpath?

Tested on macOS Monterey. 2016 MacBook Pro.

Model zoo?

It might be a good idea to create a model zoo or pipeline hub, for storing all the pipelines we create and make openly available.

That way, if people want to test one, they can download one, without downloading the rest.

Then in FP, we could add a widget for accessing the model zoo and interacting with it, adding simple ways to browse through the available models and choose which to download.

BUG: Some patches are missed during inference

@SahPet observed that the final segmentation had holes (or patches/squares) in them, where there were tissue. It seemed to happen close to the border of the tissue. This did not happen for all WSIs, but happened often enough to be of annoyance.

The result was both visualized in FP and imported and visualized in QuPath. The same holes were observed, and it is therefore not a rendering issue. It is likely not a patch generation issue, but perhaps something wrong with the stitcher.

running predictions using NoCodeSeg model

Hi
Thanks for the great software.
Fast Pathology is exiting/crashing when I try using an onnx model from here:
https://github.com/andreped/NoCodeSeg/tree/main/models

Selected file: X:/Pradeep/colon.tif
JPEGFixupTagsSubsamplingSec: Warning, Auto-corrected former TIFF subsampling values [2,2] to match subsampling values inside JPEG compressed data [2,1].
count:0
calling insert renderer
finished insert renderer
CurrentPos: 0
Length of wsiList: 1
CurrentPos: 0
Length of wsiList: 1
Model name in wrapper: Epithelium_HE_UNET_512_1P16B32FD6
Final model metadata config sent to pixelClassifier:
m[IE] = (OpenVINO)
m[batch_process] = (2)
m[batch_size] = (1)
m[class_colors] = (0,0,255;0,255,0)
m[class_names] = (Exterior;Epithelium)
m[cpu] = (0)
m[input_img_size_x] = (512)
m[input_img_size_y] = (512)
m[input_node] = (ImageInputLayer)
m[interpolation] = (0)
m[magnification_level] = (10)
m[mask_threshold] = (0.02)
m[model_name] = (Epithelium_HE_512)
m[name] = (Epithelium_HE_512)
m[nb_channels] = (3)
m[nb_classes] = (2)
m[output_node] = (Softmax_Layer_Transpose2)
m[patch_overlap] = (0.05)
m[pipeline] = (import;tissue_segmentation;batchgen;neural_network;stitch;render)
m[problem] = (segmentation)
m[resolution] = (high)
m[scale_factor] = (1.0f/1.0f)
m[task] = (Epithelium_HE_512)
m[tissue_threshold] = (70)
Final model metadata config WITHIN to pixelClassifier:
m[IE] = (OpenVINO)
m[batch_process] = (2)
m[batch_size] = (1)
m[class_colors] = (0,0,255;0,255,0)
m[class_names] = (Exterior;Epithelium)
m[cpu] = (0)
m[input_img_size_x] = (512)
m[input_img_size_y] = (512)
m[input_node] = (ImageInputLayer)
m[interpolation] = (0)
m[magnification_level] = (10)
m[mask_threshold] = (0.02)
m[model_name] = (Epithelium_HE_512)
m[name] = (Epithelium_HE_512)
m[nb_channels] = (3)
m[nb_classes] = (2)
m[output_node] = (Softmax_Layer_Transpose2)
m[patch_overlap] = (0.05)
m[pipeline] = (import;tissue_segmentation;batchgen;neural_network;stitch;render)
m[problem] = (segmentation)
m[resolution] = (high)
m[scale_factor] = (1.0f/1.0f)
m[task] = (Epithelium_HE_512)
m[tissue_threshold] = (70)
Current model: Epithelium_HE_UNET_512_1P16B32FD6
current WSI: X:/Pradeep/colon.tif
ERROR [19688] Std exception caught in Qt event handler invalid stof argument
ERROR [19688] Std exception caught in Qt event handler invalid stof argument
ERROR [19688] Terminated with unhandled exception: invalid stof argument

This is run on a Virtual machine with the following specs:

• Intel Xeon Gold 6250
• NVIDIA Quadro RTX 6000

---

Another issue is if I try importing WSIs with extension .ome-tiff , the program just exits/crashes.

Do let me know if I should be posting this issue on NoCodeSeg.

Thanks!
Pradeep

Thanks

Memory leak during inference

Memory does not seem to be freed after inference, at least not properly. This was observed using an ONNX model using both TensorRT and OpenVINO (CPU) inference engines. This also affects if you are running a model in batch mode (across multiple WSIs). The memory keeps increasing for every new WSI and eventually OOM occurs.

For TensorFlow, this has been a popular topic for quite some time. This happens as the session, where all inference and model graphs and created and live, is set globally for the entire process. A workaround in Python is therefore to perform all TensorFlow stuff in a child process (using multiprocessing), and then kill the child process after inference, which keeps the main process clean.

However, creating processed in C++ for this purpose is not viable. It is also surprising that the same (or at least a similar) memory leakage issue was observed using onnx runtime. Maybe there is something that is not freed in FAST? Not sure.

Windows 11 support

Just tested the latest release on a Windows 11, Dell XPS 13 laptop, and found that FastPathology crashed on launch.

I get three System Errors:
The code execution cannot proceed because X was not found (...).

where X is the following DLLs: MSVCP140.dll, VCRUNTIME140.dll, and VCRUNTIME140_1.dll.

Just tested a separate Python software, compiled using PyInstaller, Raidionics and works fine there. Both softwares are built on the cloud using GitHub Actions. Hence, this should be possible to fix, without the need of a separate binary for different Windows versions.

Add menu bar?

I feel like the GUI is missing a menu bar. Looks rather strange without it.
Does not need to have lots of features. Perhaps being able to close the software and open an about page is sufficient.
If we were to add the possibility to adjust settings, it would feel very natural to do is from the menu bar.

I could make an attempt.

openslide is installed under anaconda on Mac

Hi
While installing Fast-Pathogy on Mac, I encountered the error messages like below

dyld[45798]: Library not loaded: /usr/local/opt/libtiff/lib/libtiff.5.dylib
Referenced from: /Users/xinqi/Desktop/DigitalPathology/fastpathology_macos10.13_1.0.0/lib/libFAST.4.3.0.dylib
Reason: tried: '/usr/local/opt/libtiff/lib/libtiff.5.dylib' (no such file), '/usr/local/lib/libtiff.5.dylib' (no such file), '/usr/lib/libtiff.5.dylib' (no such file)

Reason: tried: '/usr/local/opt/jpeg/lib/libjpeg.9.dylib' (no such file), '/usr/local/lib/libjpeg.9.dylib' (no such file), '/usr/lib/libjpeg.9.dylib' (no such file)

Because those dylib files are installed under the anaconda directory, such as "opt/anaconda3/lib", I can copy each file to the new created directories one by one. But it is hard to solve the issues one by one.

If there any better way to let the FAST-pathology know the directories of those dylib files under the opt/anaconda3/lib?

Thank you very much

programe exits when import pictures

I have tried different types of pictures including tif and npdi and whenever I import pictures, the programe exits immediately.
Does it must run with Nivida GPU,

Environment: Windows 11 64bit with opencl installed. CPU Ryzen 3600, GPU 6700xt

Create binary release

Create a binary release for Windows and Ubuntu

Unable to read WSI if space in path

Converting segmentation.tiff to 'imagename'.tiff

This is just a query.

I am running FastPAthology v1.0.1 on Win 11.

I run a pipeline for all images in my project. Is there a way to name the .tiff result file to 'image filename'.tiff (instead of segmentation.tiff) and create all .tiff files in a single folder for the project instead of them being within subfolders for each WSI image? This will make it easier for importing the annotations into QuPath.

Thank you!

Close menu

Currently, when you open the menu, you have to click the "Close menu" inside the pop-up widget.

This is OK, but it would be more user-friendly, if it was also possible to close it by clicking the menu icon a second time. Essentially, being able to toggle.

Uninstall not properly working

Observed on Windows 10.

If you install the software, unintall it, and reinstall. The program does not launch. It is probably because the uninstall is not properly working.

However, it worked after manually deleting the models/pipelines after install. Therefore, by simply deleting these during uninstall directly, this should work.

Strange zoom behaviour

If you render a WSI and zoom as far in as you can, you will observe that you are unable to zoom out again. The rendered image just becomes black, and completely stops the user from further interacting with the image.

What is done in other softwares, like QuPath, is to have a limit on how far you can zoom in. Perhaps we should do something similar?

Stopping a pipeline does not always stop it running

Seems like if I stop a pipeline, it will remain running some times, at least for a long while.

Might be that we need a more bruteforce way to kill a pipeline which kills all child processes/workers.

Observed on Win10

mrxs file format

Thought its better to create a new issue instead of using the other one for mrxs file format
#6 (comment)

AFAIK MRXS is supported by OpenSlide (see here).

But I realized that I had forgotten to add this as an option in the selectFile method. I therefore made a commit that adds support for MRXS just now: d810332

To use the most recent version of FP (nightly, which has not yet been released). You can download the produced artifact: https://github.com/AICAN-Research/FAST-Pathology/actions/runs/1666348776

Simply download the file "Windows package", unzip, and run the installer to update your FP version.

I can open a mrxs file in FP now, but the file opens like this:

When I run predictions using the Epithelium models in NoCodeSeg, it takes a while and the end result looks like this on an inset:

However, when I export the mrxs file to a tiled-tiff using the CaseViewer software, followed by conversion in vips to a pyramidal tiff (jpeg, quality = 85), and then run predictions, I get this:

The image magnification is 10 for both.

I could share the dataset with you but will have to be via email.
We wouldn't need any agreements. Its just that I can't share it directly as its a cross-institutional project.
The owner will have to share it with you directly.

Cheers
Pradeep

An issue where only raw files are created when run with "Attribute patch-level 2 or 3"

When deploying with patch-level 2 or 3, only "segmentation.raw" file is created instead of segmentation.tiff.

In the fastpathology screen, low resolution segmentation overlays are made well. Please see the link below for photos.
https://forum.image.sc/t/question-about-converting-fastpathology-segmentation-file-for-importing-to-qupath/72855

Thank you so much,

Seoung Wan Chae.

Add option to delete result from project

Would need to update the list of which results exist in a project after this is done and update the renderer.

Issue Running on Windows or Ubuntu

Hi,

I'm Interested in this project. Attempted to run on both Windows and Ubuntu. I manually copied some fonts for Qt and I believe core dependencies are met. When running on Ubuntu, getting the following error:

The windows verbose log is:

With application crashing at this point.

Any insights on how to address.

Brian

Add documentation on how to add your own models

Add an explanation in github wiki, GUI or as a markdown file in the repo on how to create/add your own models

Wrong opening script editor behaviour

When opening the script editor it always opens a new script editor, and disregards if there is already one open.

I think it is better to have a single script editor open at a time. If we want to support multiple, I think we should make the script editor widget a little smarter, to support tabs, to switch between different documents.

Observed on Win10.

Magnification selection not working properly

Tried to run the BACH model on a x40 WSI, and it crashed due to:

Exception caught: Thread stopped because: No level close enough to magnification of 20 was found in the image pyramid.

This image plane exists, so there is something wrong.

fastpathology crashes upon running a segmentation model

I am trying to run a segmentation model on fastpathology with the following pipeline:

PipelineName "Lung Tumor Segmentation"
PipelineDescription "Segmentation of the tumors of the lung"
PipelineInputData WSI "Whole-slide image"
PipelineOutputData segmentation stitcher 0
Attribute classes "Background;Tumor"

### Processing chain
ProcessObject tissueSeg TissueSegmentation
Attribute threshold 85
Input 0 WSI

ProcessObject patch PatchGenerator
Attribute patch-size 224 224
Attribute patch-magnification 5
Attribute patch-overlap 0.0
Attribute mask-threshold 0.05
Input 0 WSI
Input 1 tissueSeg 0

ProcessObject network SegmentationNetwork
Attribute scale-factor 0.00392156862
#Attribute inference-engine TensorRT
Attribute inference-engine OpenVINO
Attribute model "/home/soroush47/Work/KaggleSeg/onnxModel/SegModel1.onnx"
Input 0 patch 0

ProcessObject stitcher PatchStitcher
Input 0 network 0

### Renderers
Renderer imgRenderer ImagePyramidRenderer
Input 0 WSI

Renderer segRenderer SegmentationRenderer
Attribute opacity 0.5
Attribute border-opacity 1.0
Input 0 stitcher 0

When I run the pipeline I get the following errors in the terminal and the program crashes.

parsing
E: [ncAPI] [    504790] [QThread] ncDeviceOpen:764	global mutex initialization failed
QObject::~QObject: Timers cannot be stopped from another thread

Zoom is not following the pointer, but the middle of the screen

Two suggestions that would be a large improvement to user experience, in my opinion:

1. Zooming is not following the position of the arrow/pointer in the image (like Qupath and many other WSI renderers), but always follows the middle of the screen. This makes it hard to navigate the image. Any way to make the zoom center around the arrow instead?

2. Also, there is now way to navigate the image with the keyboard arrow keys. Navigating a lot of WSIs with the mouse only is very demanding for the wrist and most pathologists use the keyboard arrows as well.

macOS bug

Just tried installing FP and testing it on a new macOS with Intel CPU, and for whatever reason it failed.

The machine did not have homebrew or anything like that. Hence, I set that up, but after following this instruction for installation. I got the following below:

dyld[68800]: Library not loaded: '/usr/local/opt/jpeg/lib/libjpeg.9.dylib'
Referenced from: '/Users/sigridberg/Downloads/fastpathology_macos10.13_1.0.1/lib/libFAST.4.4.0.dylib'
Reason: tried: '/usr/local/opt/jpeg/lib/libjpeg.9.dylib' (no such file), '/usr/local/lib/libjpeg.9.dylib' (no such file), '/usr/lib/libjpeg.9.dylib' (no such file)

If I tried to install libjpeg myself through brew install libjpeg FP launches, but when trying to load a simple WSI it fails and complains about mismatch in libjpeg version and what FP/FAST expects.

Have anyone observed the same?

Inference engines not available on macOS

When testing FP on macOS, it was observed that none of the inference engines were available.

I believe both TF and OpenVINO used to work before.

It would be great if at least OpenVINO worked, such that most models could be deployed for macOS users.

I would prioritize OpenVINO over TF, as most models will be converted to ONNX.

BUG: Opened WSI skipped in batch mode

@SahPet observed that when performing batch inference, the WSI that is currently opened is skipped.

This seemed to work before, but somehow skips now.

It is likely related to a recent commit where background thread inference was reverted to be run on the main thread, due to runPipeline having challenges with visualization.

runInThread crashes on macOS

Application Specific Information:
-[NSOpenGLContext setView:] must be called from the main thread.

Stacktrace:
Thread 11 Crashed:: QThread
0 com.apple.AppKit 0x00007fff2e134abc -[NSOpenGLContext setView:] + 229
1 libqcocoa.dylib 0x000000010639fb39 0x10634c000 + 342841
2 libqcocoa.dylib 0x000000010639f502 0x10634c000 + 341250
3 libQt5Gui.5.14.0.dylib 0x0000000101fa94f7 QOpenGLContext::makeCurrent(QSurface*) + 167
4 libQt5OpenGL.5.14.0.dylib 0x000000010251d954 QGLContext::makeCurrent() + 100
5 fastpathology 0x0000000100c765cb fast::ProcessWidget::runInThread(std::__1::basic_string<char,

Need to skip incomplete batches

Currently, it is possible to run batch inference using the ImageToBatch process object.

However, if we want to take advantage of the temporal dimension, for instance if your model includes the TimeDistributed layer in Keras.

To get that working with FAST, we need to do a simple trick in the model, where we transpose the batch and temporal dimensions. That way, when generating a batch of 20, we are actually using 20 patches in a bag or sequence.

However, this might fail, if the model has a fixed size on the temporal dimension, and I think it is only relevant for the last batch, which can be incomplete. I believe FAST still sends it to the network, but it might be a good idea to have an option to disregard the last batch (or incomplete batches), if it occures. To account for this potential problem.

However, in my scenario, I had trained my model with None shape for the temporal axis, so it did not prompt any errors, but if the model is trained on a bag size of a 100 patches, you might not want to give it 6 patches, as the results might be poor.

Stop button not stopping

When first opening the software (tested artifact), you are prompted whether to download and install pretrained models.

Then if you accept, you are prompted a progress widget. This widget contains a stop button, but if you try to click it,
the widget will close, but the download will still continue.

Volumes directory not found on macOS

When trying to open a WSI from an external drive, it was observed that the user was unable to access the /Volumes directory.