fnndsc / ami

AMI Medical Imaging (AMI) JS ToolKit

---

Content

1. Hello AMI
2. Features
3. Usage
4. Developer corner
5. Change log
6. Credits
7. Citations

Hello AMI

	Lesson 00: Load Load DICOM Data and get a nice Series/Stack/Frame structure.		Lesson 01: Visualize 3D Look at the data we loaded in 3D.
	Lesson 02: Mesh Add a mesh to the scene.		Lesson 03: Visualize 2D Look at the data in 2D.
	Lesson 04: Labelmap Overlays on top of you data.		Lesson 05: TRK TRK - coming soon.
	Lesson 06: Volume Rendering Volume Rendering.		Lesson 07: Lookup tables Lookup Tables - coming soon.
	Lesson 08: Custom progress bar Custom progress bars.

(more) Advanced demos

Volume rendering, 2D viewer, arbitrary reslicing and more examples and advanced demos there!

Features

✅ READY 🔶 IN PROGRESS OR LIMITED SUPPORT ❌ ON ROADMAP

Capabilities	Volumes	Meshes	Widgets
✅ 2D Visulization	✅ Dicom	✅ VTK (THREEJS)	🔶 Handle (2D/3D)
✅ 3D Visualization	✅ NRRD	✅ STL (THREEJS)	🔶 Probe (2D/3D)
✅ Volume Rendering	✅ Nifti	✅ TRK	🔶 Ruler (2D/3D)
✅ Lookup Tables	✅ MHD/(Z)RAW	✅ FSM	🔶 Orientation (2D/3D)
✅ Label Maps	✅ MGH/MGZ	❌ CURV	🔶 Angle (2D/3D)
	❌ JPEG

Usage

Pre-requisites

• ES2015 promises support. (consider using polyfills if needed)
• Load THREEJS your index.html BEFORE AMI.

<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r81/three.js"></script>

Yarn

$> yarn add ami.js

Note: you might need to include babel transforms in you build process.

// app.js
import * as AMI form 'ami.js';
window.console.log('Ready to rock!!');

ami.js

Check-out the lessons to get started quickly.

New: Use the new factory not to have to include three in index.html.

import * as THREE from 'three';
import {stackHelperFactory} from 'ami.js';

const StackHelper = stackHelperFactory(THREE);
const stackHelper = new StackHelper();

Add AMI in your index.html after THREEJS.

<!-- index.html -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r81/three.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/ami.js/0.0.12/ami.min.js"></script>
<script src="app.js"></script>

#app.js
const AMI = AMI;
window.console.log('Ready to rock!!');

Developer corner

Get the source code and related packages.

$> git clone https://github.com/FNNDSC/ami.git
$> cd ami
$> yarn install

To run examples (browserify/babelify/serve the example)

$> yarn example <examples name>

#run the geometries_slice example
$> yarn example geometries_slice

Build standalone library to lib/

$> yarn build:ami

Build minified standalone version run

$> yarn build:ami:prod

Tests

$> yarn test

Documentation

$> yarn doc

Build and deploy dist/ to gh-pages

$> yarn deploy

Find out more about the API.

Change log

Credits

AMI would not exist without them:

THREEJS

• Base components such as Vectors, Matrices and Objects3D.
• HTML template for example page.
• Author(s): mrdoob

DicomParser

• DICOM parsing relies on it.
• Author(s): chafey

CornerstoneWADOImageLoader

• Was used to figure out how to use the dicom parser properly.
• Author(s): chafey

NIFTI-Reader-JS

• Nifti parsing relies on it.
• Author(s): rii-mango

NRRD-JS

• NRRD parsing relies on it.
• Author(s): jaspervdg

JPEGLosslessDecoderJS

• JPEG Lossless Decoder for DICOM images
• Author(s): rii-mango

Image-JPEG2000

• JPEG 2000 Decoder for DICOM images
• Author(s): jpambrun, mozilla

Pako

• GZ file decompression
• Author(s): nodeca

Citations

2017

• From brain imaging to weather imaging - McCaie - informaticslab, Met Office (blog post)
• Medical imaging in the browser with the A* Medical Imaging (AMI) toolkit. - Rannou et al. - ESMRMB 2017 (poster)
• Reusable Client-Side JavaScript Modules for Immersive Web-Based Real-Time Collaborative Neuroimage Visualization - Bernal-Rusiel et al. - Frontiers in Neuroinformatics 2017 (article)

2016

• Volume Visualization Tools for Medical Applications in Ubiquitous Platforms - Arbelaiz et al. - LNICST 2016 (article)
• Interoperable communication of quantitative image analysis results using DICOM standard (DICOM4QI) - Fedorov et al. - RSNA 2016 (poster)

ami's Issues

NII support

Use the simple XTK nii parser to get started then maybe get going with Papaya.
https://github.com/xtk/X/blob/master/io/parserNII.js

add filters to the shaders

0 - not filter
1 - trilinear interpolation "filter"
...

Update Github wiki

Examples/demos organization + fiddles/pens

[feature] lut should be provided in array (vs texture) to the shaders

• one array for color
• one array for opacity

The texture has a limited resolution (currently 256 values) which ends up degrading the data visualization.

Update README

Support JPEGBaseline DICOM

Using dicomParsers, like in the wado loader:

need flat icon

Switch the whole code base to ES6

add argument to rotate to allow user to decide rotation direction

https://github.com/FNNDSC/ami/blob/master/src/cameras/cameras.orthographic.js

camera.rotate(); // clockwise
camera.rotate(-1); // counter clock-wise

in cameras.ortho

  rotate( clockwise=1) {
    this.center();

    // Pi/2 rotation if camera is positions at the front of the volume
    // else -Pi/2 rotation
    let invertClock = 1;
    if(this.position.distanceTo(this._back) < this.position.distanceTo(this._front)){
      invertClock = -1;
    }

    // Rotate the up vector around the "zCosine"
    let rotation = new THREE.Matrix4().makeRotationAxis(
      this._zCosine, 
      clockwise * invertClock * Math.PI/2);
    this.up.applyMatrix4(rotation);

    this._updateMatrices();
  }

[feature] Use maps in LUT Helpers

Store all LUTs in a map for easy access/add/remove.

Build minified ami.js

We should minify the js bundle after running through browserify

data texture

create a shader texture which will display target data over an object if location matches.

slice geometry

Create a THREEJS like slice geometry to slice a cube easily.

Add nrrd parser

https://github.com/scijs/nrrd-js

could be useful for SPL folks
https://github.com/stity/spl-brain-atlas

TRK support

Raw parser working, needs to be formatted/integrated in VJS and a demo!

Load/parser:
https://github.com/FNNDSC/vjs/blob/master/src/loaders/loaders.trk.js

Demo:
https://github.com/FNNDSC/vjs/blob/master/deprecated/examples/trk_loader/trk_loader.js

Synchronized views example

Showcase how one can connect 2 separated renderers. Useful for remote collaborators.

Lesson 01: Visualize 3D does not work.

Test for both Chrome Version 48, 64-bit and Safari.
The fiddle example could not display anything. Please take a look.

Re defining mouse inputs

Is it possible to define what each mouse input do ?
For instance, I want to change the index via mouse wheel, change ww/wc via left click instead of having them in stack helpers.
I cannot find it in any example.

EDIT: I guess I spoke too fast.

fnndsc logo

Create VJS controls

Right now, we are using tweaked THREEJS controls.

We should create medical imaging specific controls.

• for 2D orthographic camera:
  -> scroll: go through slices
• for 3D perspective camera:
  -> scroll: zoom

Allow to hook up special behavior, in XTK like way.

Widgets

See MedXT and Cornerstone

https://www.medxt.com/viewer?s=27462

http://chafey.github.io/cornerstoneDemo/

allow fine rotation of otho camera

rotate should get the rotation angle as input

this rotation angle should be the angle relative to the current rotation (or relative to the default orientation?)

reslicing example is broken

http://fnndsc.github.io/ami/#geometries_slice

The slice is not well positioned, there is an offset in a direction.

To be investigated and fixed ASAP.

VTK support

Needs to support old and new (XML) VTK formats.

http://www.vtk.org/wp-content/uploads/2015/04/file-formats.pdf

Layers shader slow *sometimes*

Sometimes it is painfully slow to scroll between slices in lesson 04 or examples label maps or image compare.

Just rebuilding AMI fixes it sometimes...

Slim down NPM dependencies

It takes forever to run npm install.

[feature] Thumbnails image from frame

Add a method to get well formatted data to display thumbnails easily.

Remove data from github

Data should be hosted in a proper location and the github repo should only contain source code.

Minimal/(very)small datasets are acceptable for testings.

2D Representation isn't working

Whenever I use a serie to make a 2D representation, following the example, I get almost half the serie that works, then the other half is just noise (like on old tvs).

Here's the view for the images that don't compute

I don't know where it comes from, as it works on the OSIRIX desktop viewer.

It's the same on the example where I load dicom files with those series.
Is there something I'm missing ?

Thanks.

Polymer viewer

Create a simple dicom view element.
Input: data = [image1, image2, image3, etc.]

add filename in frame object

it will also help cleanup the lesson 02, where 2 series are being loaded but it is hard to know which file belong to which series without it.

Bower install

People should be able to install it with bower.

2 series loaded in lesson 02

we should only load 1 series.

As a result, the stack displayed is not always the one presented in the thumbnail.

DICOM labelmaps support

Binary demo dataset from Andreii

DICOM Loader THREEJS friendly

Mimic THREEJS loading/parsing approach for DICOM.

Ideally it create a "ImageHelper" (contains geometries + textures + stackHelpers)

Terminology:
Geometry: -
Texture: -
Helper: Geometry + Texture. It is a ThreeJS Object that can be visualized.

Models and views should be THREEJS like

A THREEJS dev should be confortable with using VJS. Follow same conventions and logic.

VJS allows us to create/update Frame, Stack and Image geometries.
Same for textures.

Might also need helpers such as "ArrowHelper", etc.

1 pass ray casting

Must follow @pieper implementation as closely as possible.
(https://github.com/pieper/CommonGL/blob/master/ShaderComputation/ShaderComputation.py)

Underline goal is to extract common (python/JS) GLSL functions and get started with CommonGL.

signed data not working

In the shaders, AMI only handles unsigned data.

upload hero image for readme

add url/files helpers to loader

People shouldn't have to write the load sequences in general.

https://github.com/FNNDSC/ami-loader/blob/master/ami-loader.html

SEO lessons

sitemaps, robots.txt, canonical url, etc.

set instead of setHex to set colors in border and bbox helpers

Floats are not unpacked properly in shaders

32 bits are stored in RGBA:
https://github.com/FNNDSC/ami/blob/master/src/shaders/glsl/shaders.unpack.glsl

There should be a way to specify if we want to unpack it as integers (as it currently does), or as a float.
https://en.wikipedia.org/wiki/Single-precision_floating-point_format#Converting_from_decimal_representation_to_binary32_format

Fix Nifti volumes orientation

Right now, Nifti parser doesn't read data orientation information.

Update README

README is outdated, it should reflect current implementation, logic and goals.

Potential windows bug

examples/viewers_dicom2D/index.html three.min.js:631 THREE.WebGLRenderer 75 models.stack.js:235 do not know how to order the frames... – Hemanth Anakapalle 10 hours ago

three.min.js:704 THREE.WebGLProgram: shader error: 0 gl.VALIDATE_STATUS false gl.getProgramInfoLog C:\fakepath(167,37-79): error X4002: indefinite logarithm Warning: D3D shader compilation failed with default flags. (ps_3_0) Retrying with avoid flow control C:\fakepath(167,37-79): error X4002: indefinite logarithm Warning: D3D shader compilation failed with avoid flow control flags. (ps_3_0) Retrying with prefer flow control C:\fakepath(167,37-79): error X4002: indefinite logarithm – Hemanth Anakapalle 10 hours ago

Warning: D3D shader compilation failed with prefer flow control flags. (ps_3_0) Failed to create D3D shaders. – Hemanth Anakapalle 10 hours ago

WebGL: INVALID_OPERATION: useProgram: program not valid three.min.js:671 WebGL: INVALID_OPERATION: drawArrays: no valid shader program in use 254three.min.js:600 WebGL: INVALID_OPERATION: useProgram: program not valid three.min.js:600 WebGL: too many errors, no more errors will be reported to the console for this context. – Hemanth Anakapalle 10 hours ago

I am seeing all these errors over browser console. I am trying this over windows, Is it fine ? –

Gulp tasks are messy and slow

It takes 30+ seconds to browserify/babelify/glslify the core - it is way to much

Assets

Store here larger files not to make git source code too big

Black Box when exporting to OBJ Format

I used the Three.js obj exporter https://github.com/mrdoob/three.js/blob/master/examples/js/exporters/OBJExporter.js

and this code to download the file `

var exporter = new THREE.OBJExporter();
var result = exporter.parse(scene);

var link = document.createElement('a');
document.body.appendChild(link); 
save( new Blob( [ result], { type: 'text/plain' } ), 'model.obj' );

function save( blob, filename ) {

    link.href = URL.createObjectURL( blob );
    link.download = filename || 'data.json';
    link.click();

    // URL.revokeObjectURL( url ); breaks Firefox...

}`

and i get a black box