digitalslidearchive / histomicstk

A Python toolkit for pathology image analysis algorithms.

HistomicsTK

HistomicsTK is a Python package for the analysis of digital pathology images. It can function as a stand-alone library, or as a Digital Slide Archive plugin that allows users to invoke image analysis jobs through HistomicsUI. The functionality offered by HistomicsTK can be extended using slicer cli web which allows developers to integrate their image analysis algorithms into DSA for dissemination through HistomicsUI.

Whole-slide imaging captures the histologic details of tissues in large multiresolution images. Improvements in imaging technology, decreases in storage costs, and regulatory approval of digital pathology for primary diagnosis have resulted in an explosion of whole-slide imaging data. Digitization enables the application of computational image analysis and machine learning algorithms to characterize the contents of these images, and to understand the relationships between histology, clinical outcomes, and molecular data from genomic platforms. Compared to the related areas of radiology and genomics, open-source tools for the management, visualization, and analysis of digital pathology has lagged. To address this we have developed HistomicsTK in coordination with the Digital Slide Archive (DSA), a platform for managing and sharing digital pathology images in a centralized web-accessible server, and HistomicsUI, a specialized user interface for annotation and markup of whole-slide images and for running image analysis tools and for scalable visualizing of dense outputs from image analysis algorithms. HistomicsTK aims to serve the needs of both pathologists/biologists interested in using state-of-the-art algorithms to analyze their data, and algorithm researchers interested in developing new/improved algorithms and disseminate them for wider use by the community.

HistomicsTK can be used in two ways:

• As a pure Python package: enables application of image analysis algorithms to data independent of the Digital Slide Archive (DSA). HistomicsTK provides a collection of fundamental algorithms for tasks such as color normalization, color deconvolution, nuclei segmentation, and feature extraction. Read more about these capabilities here: api-docs and examples for more information.

  Installation instructions on Linux:

  To install HistomicsTK using PyPI:

  $ python -m pip install histomicstk --find-links https://girder.github.io/large_image_wheels

  To install HistomicsTK from source:

  $ git clone https://github.com/DigitalSlideArchive/HistomicsTK/
  $ cd HistomicsTK/
  $ python -m pip install setuptools-scm "Cython>=0.25.2" "scikit-build>=0.8.1" "cmake>=0.6.0" "numpy>=1.12.1"
  $ python -m pip install -e .

  HistomicsTK uses the large_image library to read content from whole-slide and microscopy image formats. Depending on your exact system, installing the necessary libraries to support these formats can be complex. There are some non-official prebuilt libraries available for Linux that can be included as part of the installation by specifying pip install histomicstk --find-links https://girder.github.io/large_image_wheels. Note that if you previously installed HistomicsTK or large_image without these, you may need to add --force-reinstall --no-cache-dir to the pip install command to force it to use the find-links option.

  The system version of various libraries are used if the --find-links option is not specified. You will need to use your package manager to install appropriate libraries (on Ubuntu, for instance, you'll need libopenslide-dev and libtiff-dev).

  To install from source on Windows:

  1- Run the following:

  $ pip install large-image
  $ pip install cmake
  $ git clone https://github.com/DigitalSlideArchive/HistomicsTK/
  $ cd HistomicsTK/
  $ python -m pip install setuptools-scm "Cython>=0.25.2" "scikit-build>=0.8.1" "cmake>=0.6.0" "numpy>=1.12.1"

  2- Run pip install libtiff

  3- Run pip install large-image-source-tiff to install typical tile sources. You may need other sources, which would require other libraries.

  4- Install Visual Studio 15 2017 Community Version

  5- Install C++ build tools. Under Tools > Get Tools and Features ... > Desktop Development with C++, ensure that the first 8 boxes are checked.

  6- Run this:

  $ python -m pip install -e .
  $ pip install girder-client

  To install from source on OSX:

  Note: This needs to be confirmed and expanded by an OSX user. There are probably assumptions made about available libraries.

  Use homebrew to install libtiff and openslide or other libraries depending on your desired tile sources.

  Run:

  $ python -m pip install histomicstk large-image-source-tiff large-image-source-openslide

• As a image-processing task library for HistomicsUI and the Digital Slide Archive: This allows end users to apply containerized analysis modules/pipelines over the web. See the Digital Slide Archive for installation instructions.

Refer to our website for more information.

Previous Versions

The HistomicsTK repository used to contain almost all of the Digital Slide Archive and HistomicsUI, and now container primarily code for image analysis algorithms and processing of annotation data. The deployment and installation code and instructions for DSA have moved to the Digital Slide Archive repository. The user interface and annotation functionality has moved to the HistomicsUI repository.

The deployment and UI code will eventually be removed from the master branch of this repository; any new development on those topics should be done in those locations.

Funding

This work is funded by the NIH grant U24-CA194362-01.

See Also

DSA/HistomicsTK project website: Demos | Success stories

Source repositories: Digital Slide Archive | HistomicsUI | large_image | slicer_cli_web

Discussion: GitHub Discussion | Discourse forum

histomicstk's Issues

EmbedBounds shape check incorrectly checks image planes

Shape check should only confirm that inputs have same number of rows/columns.

Installing ITK with python as a package for ubuntu

This is ITK v4.7 and I just verified on Monday that this works.

https://github.com/zachmullen/girder_training/blob/master/ansible/vagrant_playbook.yml#L17-L23

Then you can apt-get install insighttoolkit4-python

Implement AdaptiveColorNorm

Currently this is empty.

Installer is trying to download matplotlib each time I run setup.py

Implement a CLI that inputs a wholeslide image, segments nuclei, and generates features

Develop front-end for DSA

The front end should have ways to:

• select data from girder
• viewer for visualizing whole-slide images and annotations/results
• select and run analysis CLIs exposed in the server directory of histomicsTK

SparseColorDeconvolution output intensities are inverted

Alpha expansions w/ graph coloring - implementation

Refinement of label images using multi-label pygco graph cuts (alpha-expansions or alpha-beta swap). Graph coloring is used to reduce the computational burden so that # objects < # cells.

Fix optimization routine in SimpleMask.py

Optimization routine 'fmin_slsqp' from scipy is stopping early before convergence. Need to examine stopping criteria in documentation and do some more extensive testing for constrained curve fitting of image histograms.

Remove condensing from label processing functions

Functions for processing label images automatically condense the label image values to fill gaps for simplicity. This breaks the correspondence between the values of objects before and after calling the function. This can be fixed by skipping over 'empty' labels in the input.

Refactor histomicstk into submodules and correct naming

Issues include moving functions into submodules, converting function names from snake-case to camel-case, correcting function names in "See Also" document strings, and adding underscore prefixes to non-exposed functions.

Implement CLI for segment nuclei and generate annotations for them

Cycle in MaxClustering

There is a cycle in trajectory tracking for MaxClustering that causes an endless loop. Need to check the edge values of tracking matrices for correctness.

Convert function outputs to named tuples

Rendering of annotations in HistomicsTK front-end

This issue is about adding the functionality necessary for rendering annotations on top of images in the HistomicsTK front-end.

@manthey and @jbeezley. Can one of you please take this up?

You can use some dummy annotations of each type to build the functionality necessary to render them.

Address memory limits in Sample.py

Sample attempts to load a low-res version of the entire slide to generate a tissue mask. If a 1.25X magnification is not provided by the WSI then this could result in loading an entire slide at 5X or high magnification.

Modify Auto CLI REST endpoint code to run standalone CLIs using Docker in girder_worker

Optimize memory in label image functions

Examine in-place versus copy operations.

MaxClustering - fix seeding

Deal with cases where there are multiple connected seed pixels in (Response == Max) - typically encountered where Response is flat.

Check matching input sizes in EmbedBounds

EmbedBounds does not throw error when mask/label and intensity images are not same size.

Write code to automatically create REST routes for slicer execution model CLIs

Remove dependency on matplotlib

I don't think we need to require matplotlib for installing histomicstk. This is causing issues installing on our cluster as it tries to pull in all kinds of other packages. People are free to install this independently and there are other options for viewing images.

Bump girder, girder_worker and large_image in HistomicsTK/.travis.yml

This issue is about bumping hash of girder, girder_worker and large_image to the latest commit in HistomicsTK/.travis.yml

Clarify parameter names in EmbedBounds

Create an API for getting tiles of whole-slide images

A good way to do this is by creating an abstract WholeSlideImage class which defines the API and then make derived classes of it such as GirderWholeSlideImage (which serves an image stored on girder) and LocalWholeSlideImage (which serves an image located on the local machine).

We would want all analysis functions in histomicstk to take objects of the abstract WholeSlideImage class as input instead of a numpy array or PIL image.

Fill values distort model fitting procedures

The fill values used when requesting image tiles from OpenSlide can produce errors in building models to distinguish tissue from glass. Need to modify code to exclude pixels from filled areas that are added to create an integral-sized tiling.

Pass values of CLI parameters in json.dumps format while running a CLI through front-end

While running a CLI through the histomicstk front end (localhost:808/histomicstk#<cli-name>), pass values of CLI parameters in the POST Request are in the format outputted by json.dumps. I use json.loads to convert them back to a value of their python type.

Use comma separated values for vector parameters in a CLI

@jbeezley
In the front-end, use a single text box for vector parameters (integer-vector, float-vector, double-vector, string-vector) in a CLI and expect the user to provide a comma separated list of values

Upgrade to GeoJS 0.9

This contains zoom animations, which are an important feature.

Fix the generation of API documentation on ReadTheDocs

Slicer XML based command-line argument parser

Create a slicer xml based command-line argument parser using argparse to enable developers to write slicer execution model CLIs in python.

Once this is done developers will be able to write CLIs in python once which can then be run both on command-line (the in-line python script CLIs that we have now wont allow this) and called over the web via REST end point.

Easy setup of DSA stack using vagrant

@brianhelba can we assign this to someone? This will allow everyone to use the same development environment.

RAGLayer error when number of objects <= 2

Load images via large_image API

Create a stable branch for HistomicsTK and host it on girder.neuro.emory.edu

The stable branch should contain the most recent stable version of HistomicsTK that we can show to people.

Optimization fails in SimpleMask.py

On image TCGA-S9-A6UA-01Z-00-DX1.99AB8786-858B-46A0-94BA-C5AA1CD5351B.svs

File "/opt/lib/python2.7/site-packages/histomicstk-0.1.0-py2.7.egg/histomicstk/SimpleMask.py", line 55, in SimpleMask
TissuePeak = Peaks[yHist[Peaks[1:]].argmax()+1] #take highest peak among remaining peaks as background
ValueError: attempt to get argmax of an empty sequence

Poisson mixture model cuts

Implement poisson mixture modeling and binary pygco graph cuts for foreground/background segmentation.

Add types missing in girder_worker to support translation from slicer CLIs

3rd party nuclear segmentation with pygco

Implement end-to-end nuclear segmentation pipeline using the following components:
-Whole-slide Reinhard normalization
-Poisson mixture models / binary graph cuts
-Constrained log (cLoG) splitting to produce label image
-Multilabel graph cuts refinement to refine label image

Setup CMake based testing and add some documentation

@brianhelba You can add the documentation for testing here. I will find a good place to move it to when we organize our documentation later.

Boundary and feature formats

Define the formats to be produced by object segmentation and feature extraction.

Boundaries - will need a function to convert a list of 2 x N arrays (x,y) to some format for consumption into Girder.

Features - how to link the feature names to the rows of a K x N array. Would these be consumed into Girder or kept as arrays on disk?

Install requirements_c.txt also in .travis.yml

@brianhelba should we also be installing requirements_c.txt in .travis.yml.

I had to separate all packages that depend on c libraries into a separate file for ReadTheDocs to work. The issue is described here. I had to add mocks for them in docs/conf.py and exclude them from install_requires in setup.py to get the documentation to build without failure on ReadTheDocs. Dont know if there is a better way?

Optimize calls to scipy.ndimage.measurements.find_objects

Avoid unnecessary computation by calling with masks for select objects.

Sample.py produces empty output

On images:
TCGA-HW-8322-01Z-00-DX1.B6F32F6B-7FA3-42E4-98CB-F2F55208F7C4.svs
TCGA-HT-7479-01Z-00-DX1.E310974D-E52C-4634-8E16-2A41E8C37D45.svs
TCGA-DU-8165-01Z-00-DX1.8d633ff1-6aed-41e5-8518-8856dbe4a718.svs
TCGA-DU-8166-01Z-00-DX1.82951397-8c63-4c0f-8696-f081b170e21f.svs

File "/opt/lib/python2.7/site-packages/histomicstk-0.1.0-py2.7.egg/histomicstk/Sample.py", line 119, in Sample
Pixels = np.concatenate(Pixels, 1)

Fix issue with ConvertSchedule.py and 40X slides

ConvertSchedule is generating bad size conversions for some slides, causing Sample.py to throw errors.

Fix handling negative findings in MinimumModel contour seeding

Fix seeding of elongated cells in GaussianVoting.py

Elongated cells are not being detected properly by GaussianVoting. Their symmetry is more axial than radial, and the votes are too diffuse to be detected as seed locations.

Initial feature set - implementation

Switch from romanesco to girder_worker

@brianhelba .travis.yml needs to be updated to switch from romanesco to girder_worker

Max clustering - implementation