candelbio / mantis-viewer

Electron-based multiplexed imaging viewer

Mantis Viewer

Mantis Viewer is an application developed by the Parker Institute for Cancer Immunotherapy for viewing and analyzing multi-channel pathology imaging, such as IHC, Vectra, MIBI, IMC, CODEX, or other technologies. It has been designed to be highly performant and responsive when analyzing both large and small images and to support workflows with segmentation data.

Using Mantis Viewer

If you want a walkthrough for using Mantis Viewer, you can check out the documentation!

Downloading

If you just want to use the application head to the releases page! Otherwise continue reading if you want to build the project yourself or to contribute to this project.

Develop and Run Locally

To get up and running for the first time first clone the repository and install the dependencies. You may need to install the build dependencies for canvas first.

npm install

Once NPM has installed all of the dependencies, you can start the application.

npm start

Webpack will automatically rebuild the application and restart Electron if you are actively developing.

Running tests

Running tests is a little funky. It isn't possible to install a version of Node locally that has the same API version as the version of Node that's bundled with Electron. Since this project makes use of native modules, which have to be compiled targeting a specific Node API version, we have to reinstall or recompile the native modules before running tests so they can run using the local Node version, and after running tests so that the modules can be used with Electron's Node version. There's probably a better way to do this, but for now these are the three commands that you need to run tests locally.

npm rebuild
npm run test
npm run postinstall

Generating executables

To generate executables on all systems, you will first need to make sure all dependencies are installed

npm install

Once this is done, you can build and package the application for distribution

Linux:

npm run dist-linux

Mac:

npm run dist-mac

Windows:

npm run dist-win

When this completes, you should have executables built in the dist directory.

Creating a New Release

New releases are automatically built on CircleCI and then published to GitHub as drafts. npm-version is used to bump the version.

For a prerelease version run the following command.

npm version prerelease --preid=rc

Preminor, minor, and major releases do not need any additional arguments for npm-version.

To specify the verion during a release run the following command

npm version <new-version>

Technologies

Mantis Viewer is build using Electron, Typescript, React, MobX, and PIXI.js.

Cite Us

Please cite Mantis Viewer using the following citation:

Robert Schiemann, Lacey Kitch, Pier Federico Gherardini, Robin Kageyama, & Mike Travers. Mantis Viewer. Zenodo. http://doi.org/10.5281/zenodo.4009579

You may also cite the specific version of Mantis that you used instead if you wish. You can get citations for specific versions on the Mantis Viewer Zenodo Page

License

Mantis Viewer is distributed under Apache 2 license. See the LICENSE file for details.

mantis-viewer's Issues

Some graphics and sprites not rendering after WebGL context loss

Hello!I saw your problem in pixijs forum, maybe i have the same problem as you. Can you tell me how you fix that problem? Thanks a lot!

Add option to control image blurring

It looks like the image data is being blurred somehow when it's displayed in Mantis. Is there a way to control this behavior, or if not, how much work would it be to add a tunable parameter?

Scroll drag resets when mouse leaves canvas

When using the mouse to drag the viewing pane around, if the mouse leaves the pane, the scroll resets back to the original location before scrolling started, rather than just stopping the scroll at the current location

Screen.Recording.2022-05-18.at.12.44.29.PM.mov

Issue with selecting region of interest on image using mouse

When I try to highlight a region of interest using my mouse as described ( https://mantis.parkerici.org/populations/ ), the lasso tool never lets go, even after I release the mouse button, and the region is never selected. I did not have this issue earlier today, but now I have this issue recurrently even after re-downloading mantis-viewer. Do you have any tips on how to resolve this issue?

Zoom scroll is too slow

The default setting for the scroll wheel to zoom in and out is a bit too slow, as it takes a long time to zoom in on a large image. It would be nice if this was made a bit more sensitive, or if the user could modify this parameter

Can't import images on Mac

Hi,

I would be very interested in using mantis viewer for multipled imaging projects. I've tried the two most recent versions, but on my M1 MBP I cannot get mantis to open any images. I've screenshot the issue below. It begins saying 'loading image data' but then I get just a blank screen. When the blank screen occurs, it came up with on the console "Devtools cannot connect" or something to that effect. Wondering if you could point me in the right direction?

Thanks

Paul

replace loaded segmentation mask

It's totally possible that I'm missing something, but it appears that reloading a new segmentation mask when a previous one has already been loaded currently doesn't do anything. It gives a warning about losing previously gated populations, but I don't believe the new one gets loaded.

Is there a way to first remove an existing segmentation mask prior to reloading a different one? This is for comparing multiple different segmentations on the same image set.

Image moves when turning off all channels

When all of the channels are turned off, the entire image shifts slightly down. This is due to the No markers visible text that gets placed on the top of the display. However, when trying to toggle image channels on and off, this results in the populations shifting slightly, which makes it harder to map up marker expression with a specific population.

If the text was placed elsewhere, or if the whitespace for the text was always there, independent of the text being displayed, it wouldn't affect the layout of the canvas.

Screen.Recording.2022-05-02.at.3.09.49.PM.mov

Validate inputs for populations import

The current import populations functionality doesn't raise an error message if the wrong column names are supplied, and Mantis freezes.

Fix seed for population color mapping

Currently, each image in a dataset will have a different mapping between populations and colors. It would be nice if this was set once, project-wide, to enable consistent colors across images.

Can't import regions from tiffs without segmentation masks

I ran into an issue today where I was trying to load a new project that had spatial data encoded as a regions file. When going to the load new project interface, if I specified a regions tiff for the project without a segmentation mask, the image data loaded but not the regions. Same thing for loading a regions file in an individual image without a segmentation mask: it would accept the file, but then nothing happened. It was only if there were already segmentations loaded that the regions could get loaded in.

Not sure if this is intentional, and if the thought is that regions should define regions of segmented cells, or accidental based on the hierarchy of the data structure. Either way, having an error message pop up to explain would be helpful, or updating the documentation.

Creating Stacked Multi-page TIFFs

Hello, I'm attempting to convert a folder of $n$ TIFFs into a folder containing 1 stacked multi-page TIFF, however I'm unable to set the channel.target property of the ImageDescription with the python package Tifffile.

Currently I am trying to set a metadata field with the channel.target with the following:

_metadata = {
    "axes": "CYX",
    "channel": {"target": fov.channels.values.tolist()},
}
with TiffWriter(multi_tiff_file_path) as tiff:
    tiff.write(
        data=fov.values,
        photometric="minisblack",
        compression=_compression["algorithm"],
        compressionargs=_compression["args"],
        metadata=_metadata,
    )

In addition I have also tried converting a folder of TIFFs to an OME-TIFF but that also does not read the channel names. Do you have any suggestions on how to set the channel names for multi-page TIFFs?

Restarting on error turns into an infinite loop.

In beta.6, when Mantis restarts on an error it can turn into an infinite loop of restarts. See example project shared by Yuval.

Only display error message if populations not successfully imported

I got the following message about not all features being imported. It looks like it's displayed even if all of the cells were successfully imported

Pixel-level populations can be hard to see

The colormap that is used for displaying pixel populations includes some colors that are quite dim. This can make it hard to see where on the image the given population is. For some reason, this doesn't seem to be happen with the cell-level populations. Maybe something about the brightness settings for cell segmentation populations is different, making them easier to see?

How is max image size determined?

How is the maximum size before downsampling applies set?
I'm currently trying to look at 7 different channels, each of which are 5-15 MBs.

However, they are getting downsampled. It seems like these could all pretty comfortably fit into memory. Is there a way to increase the amount of available memory? Alternatively, what is the current max? That way I can split the file into pieces that are as large as can fit.

Images not importing on Ubuntu

I downloaded mantis-viewer_1.2.2-rc.6_amd64.deb and installed it on an Ubuntu 16.04.7 LTS machine. When I tried to import images using File -> Open -> New Project or File -> Open -> Image, I was able to select the correct directory, but when I click "Import", it takes me back to the Welcome page (where I see the "Welcome to Mantis Viewer!" message) and the images don't seem to be imported.

Switching channel mappings can cause crashes.

See example project shared by Yuval.

Disappearing Channels

Changing a channel to a different color causes them to disappear (not rendered).
Not by changing the color of the current channel, but by removing the channel and then selecting it again from a different color.
Also occurs when changing the channel mappings.
Reloading the image by changing the selected image back and forth re-renders the channels.
This is present in 1.2.3-alpha.9 and not in 1.2.3-alpha.8
Thus, this bug was probably introduced somewhere when fixing the bug where moving max channel intensity slider to min value prevented that channel from being rendered.

add inset for zoom location

It would be great if there was a little window at the top, similar to FIJI, to show where in an image the current view is when zoomed larger than 100%.

See attached for an example in upper left-hand corner of FIJI

Javascript uncaught exception on `npm start`

The following stack trace occurs when attempting to run mantis locally using npm start:

A JavaScript error occurred in the main process
Uncaught Exception:
Error: Module build failed (from ./node_modules/ts-loader/index.js):
/home/matt/github/mantis-viewer/node_modules/typescript/lib/typescript.js:139
    for (let i = startIndex ?? 0; i < array.length; i++) {
                             ^

SyntaxError: Unexpected token '?'
    at wrapSafe (internal/modules/cjs/loader.js:915:16)
    at Module._compile (internal/modules/cjs/loader.js:963:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at require (internal/modules/cjs/helpers.js:85:18)
    at Object.<anonymous> (/home/matt/github/mantis-viewer/node_modules/ts-loader/dist/utils.js:6:20)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at ./main.ts (/home/matt/github/mantis-viewer/app/main.js:10:7)
    at /home/matt/github/mantis-viewer/app/main.js:21:43
    at Object.<anonymous> (/home/matt/github/mantis-viewer/app/main.js:23:12)
    at Module._compile (node:internal/modules/cjs/loader:1118:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1173:10)
    at Module.load (node:internal/modules/cjs/loader:988:32)
    at Module._load (node:internal/modules/cjs/loader:829:12)
    at c._load (node:electron/js2c/asar_bundle:5:13343)
    at loadApplicationPackage (/home/matt/github/mantis-viewer/node_modules/electron/dist/resources/default_app.asar/main.js:110:16)
    at Object.<anonymous> (/home/matt/github/mantis-viewer/node_modules/electron/dist/resources/default_app.asar/main.js:222:9)

This is after the following npm installations:

npm install canvas
npm install

npm version: 8.5.1
node version: v12.22.9

Export images without labels

Dear Mantis team,
I have found your software useful. But when i export the png image, the labels are always present at the left top (see image attached). Is there a way to export the images without these labels? because it's taking considerable amount of area in the image and it's unnecessary

thanks,
Kedar

Resize display when zoomed

It would be nice if Mantis would resize the display to better fit the available screen space once zoom has been activated. If non-square images are loaded, only a small amount of the available screen space is used to display the image, as see in the screenshot below

add option to hide non-highlighted segmentation boundaries

Currently there is the option to give specific cell populations a color, and overlay that color over the segmentation. However, there doesn't appear to be a way to turn off the display for non-selected cells.

It would be great to add a toggle that removes the default rendering of segmentations, and only keeps those that have been given a specific color

Provide names for regions

Currently there is no way to provide metadata for the regions supplied in a file. It would be great if there was a way to provide a csv linking each integer in the regions tiff to a descriptive name, so that they don't need to be manually edited in each image in the project.

Add option for pixel populations tooltip

One really nice option when looking at segmented data is the tooltip that pops up at the top showing what population the moused-over cell belongs to:

This is helpful when there are tons of different colors, because it makes it easy to identify the cell phenotype, rather than needing to mouse over each individual population until the right one lights up.

It would be nice to have this same option when mousing over individual pixel-level populations:

This would allow the user to quickly figure out the label of a pixel population of interest.

Some potential problems:

• There might be performance issues due to the large number of pixels
• It might be challenging to have an attractive UI due to the tooltip constantly changing as the mouse is moved even slightly, due to the small size of individual pixels. For our use case, we tend to have pixels in the same neighborhood that have the same population label, but I imagine this might not be true for everyone.

Definitely not a necessity, more of the nice-to-have category

demo data

Do you have demo data I can use?

I tried loading a multi-channel tiff but nothing happened?

Zoom sticks on images

I'm running into an issue with the zoom functionality. It will stick when I'm trying to zoom in or out. I've attached a movie to explain. I'm using the scroll wheel on the external mouse for my macbook pro.

It seems like it happens more often when it's near the border of the image, but it's not 100% consistent
https://user-images.githubusercontent.com/13770365/165396478-c48c7043-fdcf-419b-9b19-bad82cc9d55a.mov

Loading image stacks

Dear Lacie and Team,

CODEX data often comes in an image stack form (.tif) that contains all channels in one file. Would it be possible to support importing the stacks / montages?

Thanks!
Zina

cannot load images or project with mantis in ubuntu 22.04

Hi team,

Problem description

I cannot open project or images in ubuntu 22.04 using the latest appimage (1.2.3-alpha.3) or the previous appimage or .deb installations (v1.2.4-alpha.2). The program launches and shows the welcome pop-up. I can click on new project or new image and correctly select them in my file system, and then the new project pop-up shows where I can define my project further (see screenshot).

When I then click 'import', I return to the welcome screen instead of seeing my project.

Troubleshooting steps undertaken

• tried different versions and installation formats (see above)
• The v1.2.4-alpha.2 version on windows is able to correctly open and show my images.
• ran via command-line, which gives the following output:

16:09:29.820 › APPIMAGE env is not defined, current application is not an AppImage
16:09:29.823 › checkForUpdatesAndNotify called, downloadPromise is null

neither of which are errors.

System information

Ubuntu: 22.04
nvidia driver 535

I would very much like to use mantis. Thank you very much in advance for your help!

default scaling for low-count data

When loading MIBI data, the default scaling is always much too low. This is fairly low priority, but it would be nice if the guess for a cap value worked better for data with very low (1-20) counts per pixel.

Importing segmentation features stalls for multiple features in large projects

Just started playing around with importing pre-computed segmentation statistics, the feature is awesome! Being able to mouse over a cell and see the computed features is super useful for downstream analysis.

Importing a CSV with FOV, label, and 1 feature is basically instant. However, importing a CSV with 4 features stalls, and I have to restart mantis. It seems to be related to larger projects: using a project with only 2 images, it works fine, but the one I currently have contains 18 images. Maybe there's some sort of nested for loop that's causing poor scaling?

Tooltip doesn't display segmentation region for multiple images in project

When multiple images are loaded as part of a project, only the first image will display the tooltip showing which region the segmentation has been assigned to. After switching to a new image, the regions no longer show up in the tooltip, only the segmentation id

Screen.Recording.2022-05-08.at.1.58.08.PM.mov