aaboyles / microbetrace

The Visualization Multitool for Molecular Epidemiology and Bioinformatics

MicrobeTrace

The Visualization Multitool for Molecular Epidemiology and Bioinformatics

Developed By (some folks at) CDC.

To Use MicrobeTrace:

• Launch MicrobeTrace
• Download Sample Data
• Download Sample COVID Data
• View the Help Wiki
• Read the Latest Manual
• Watch and follow along with the webinar
• Watch how to use MicrobeTrace for COVID-19
• Cite MicrobeTrace

To Spread the Word:

• View the MicrobeTrace Flyer
• Read our Reports
• Connect with our User Community

To Help Us Build:

• Report Bugs or Request Features
• Request Access to Our Slack
• Deploy Your Own Instance of MicrobeTrace
• Help us Develop MicrobeTrace

Public Domain

This project constitutes a work of the United States Government and is not subject to domestic copyright protection under 17 USC § 105. This project is in the public domain within the United States, and copyright and related rights in the work worldwide are waived through the CC0 1.0 Universal public domain dedication. All contributions to this project will be released under the CC0 dedication. By submitting a pull request you are agreeing to comply with this waiver of copyright interest.

License

The project utilizes code licensed under the terms of the Apache Software License and therefore is licensed under ASL v2 or later.

This program is free software: you can redistribute it and/or modify it under the terms of the Apache Software License version 2, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the Apache Software License for more details.

You should have received a copy of the Apache Software License along with this program. If not, see http://www.apache.org/licenses/LICENSE-2.0.html

Privacy

This project contains only non-sensitive, publicly available data and information. All material and community participation is covered by the Surveillance Platform Disclaimer and Code of Conduct. For more information about CDC's privacy policy, please visit http://www.cdc.gov/privacy.html.

Contributing

Anyone is encouraged to contribute to the project by forking and submitting a pull request. (If you are new to GitHub, you might start with a basic tutorial.) By contributing to this project, you grant a world-wide, royalty-free, perpetual, irrevocable, non-exclusive, transferable license to all users under the terms of the Apache Software License v2 or later.

Before you get started, please read the Developer's Guide to MicrobeTrace.

All comments, messages, pull requests, and other submissions received through CDC including this GitHub page are subject to the Presidential Records Act and may be archived. Learn more at http://www.cdc.gov/other/privacy.html.

Records

This project is not a source of government records, but is a copy to increase collaboration and collaborative potential. All government records will be published through the CDC web site.

Notices

Please refer to CDC's Template Repository for more information about contributing to this repository, public domain notices and disclaimers, and code of conduct.

microbetrace's Issues

Infer Directionality from Phylogenetic Tree

We had an idea awhile ago to infer the directionality of any two nodes by comparing the distance between each node and the two nodes latest common ancestor. Let's implement it!

Move Network to Canvas?

The network lags when it's really big and/or dense. This make sense, and it forces the browser to track >10,000 DOM elements in SVG. We could take a load off by doing it in canvas, but we'd have to re-write almost everything in the Network code (the common code and other views would continue without a problem).

Set Map to respect Node Visibility and updates

Sparkline Selector

Instead of the annoying numeric spinner, let's write a little D3 histogram sparkline with a selection event that exposes the filter value to the relevant entities.

Visual Multi-select Nodes from 2D Network

See: CDCgov#3

Map node attribute to Color in 3D View

Same as we do it in the 2D Version!

Add a message for IE users

Let them know that MicrobeTrace cannot and will never work on IE, and to try using Chrome, Firefox or Edge

Export Flow Diagram as Image

Default Link Threshold to 1/5 of available range

Live Update Flow Diagram as Data Change

Use Vue?

Move Export formats to their respective Views

I wanted to centralize exporting, but we need a more flexible approach (where every view exports logically-similar filetypes). So, CSVs go to Table View, FASTA goes to Sequence Viewer, etc.

Put Map Stuff on Canvas?

We've shied away from making the map too detailed, in part because of performance concerns. However, as our experience with the Aligner mini-game has shown, we can gain a lot of horsepower by transitioning to canvas. Accordingly, I propose we use Canvas to render the background elements of the map, and SVG to overlay the nodes and edges.

If MicrobeTrace needs to retool in the future to accomodate an additional order of magnitude of data, we can investigate doing the whole thing in canvas, like so.

Collapsible Clusters in 2d Network

Gephi Importing

Using https://github.com/Yomguithereal/gexf

BUILD SYSTEM

WebPack so it's compact and Babel so we can build it to target IE10. Or 11. Or at least Edge.

Scatterplot and Histogram Views don't export axis labels

Timeline Interoperability

Make the timeline interoperable with the app. In particular, make it playable so stuff appears at the "time" specified by the data.

Compress Save Files

Factor of 10 reduction in size!

Save/Load Session

It's busted. Fix it!

Export Table(s) as Excel

Export Sequences from Sequence Viewer

FASTA and MEGA are low-hanging fruits, here. Possibly others in the future?

Flexible Number of Attributes in Flow Diagram

Integrate Census tract data into Map

We can manage that, right?

Option to Highlight Neighbors on Hover in 2D Network

Per request from MD.

Add color-by-attribute to 3D network Links

Installation Instructions

Like this: https://stackoverflow.com/questions/13509736/javascript-create-desktop-link-programmatically/13510297#13510297

Add Size-by-attribute to 3D network nodes

Add a tile layer to the map if browser is connected to the 'net

Distance Matrix Population Heuristic

We may be able to improve compute times by implementing a cheap heuristic algorithm to decide whether we need to compute a dyad's TN93 (which is costly). Here's how it would work:

1. compute the consensus sequence.
2. compute the TN93 between each sequence and the consensus.
3. for each dyad (a, b): compute tn93 iff |a->consensus - b->consensus| <= threshold
4. return computed tn93s and render network
5. (in background) compute remaining tn93s
   (Thanks @Sergey-Knyazev for designing this algorithm)

Edge Labels in 2d Network

See CDCgov#154

MST computation is broken

Progressive Rendering

For large analyses, it might be better to render progressively:

1. launch the network view
2. kick off the distance matrix computation
3. every 1000 links (or whatever), post a message back to the browser with the new links
4. have the browser decide on the new threshold and render the links accordingly

Geospatial node visibility filtering

Set Node Colors in Phylogenetic Tree

http://phylocanvas.org/v1.x/docs/code_samples.html#cs_colorNodes

Export Graph Formats

E.g. Cytoscape, Gephi, etc... CDCgov#40

Add Color-by-attribute to 3d Network Nodes

Node Selection from Phylogenetic Tree?

Draggable Modals

The modals can be a little, uh, in the way. It would be nice to be able to drag them to non-blocking regions of the screen.

Phylogenetic tree to respect node visibility and updates

Create build process using nativifier

https://github.com/jiahaog/nativefier

Add a Version identifier somewhere visible in the app

Spin out the WMT Network Engine

WMT is now sitting on a powerful and featureful network analytics engine. It doesn't have the featureset to complete with JSNetworkX yet, but it could.

Export 3D Graph as Image

Shortest Path Pruning Heuristic

We could build an alternative to MST and threshold-based link filtering. Here's how @Sergey-Knyazev would do it:

1. (I don't remember!)

Export Plotly Graphs as Images

Node Selection from Map

Timeline overhaul

Mostly just make it work.

Link Origin-based directionality

I.E., an option to show directionality from Link lists, but hide it from links inferred by genetic proximity.

Per Request from MD.

Add Layout Modes

Since MicrobeTrace is gearing up to compete with MicroReact in terms of MicroReact's own features, it would be particularly instructive (and appropriately cheeky) to add a MicroReact View Mode, which emulated MicroReact by placing a Map in the top-left, a Phylogenetic Tree in the top right, and a timeline across the base of the screen. We could look at emulating other systems in the same way.

Web Workers for Data Parsing

Y'know, so the browser doesn't block repaints.