stevekm / igv-snapshot-automator

Script to automatically create and run IGV snapshot batchscripts

NOTE

I recommed checking out igv-reports which has more features and active development before attempting to use this IGV-snapshot-automator.

Also check out BamSnap for IGV-like bam file visualization; https://github.com/parklab/bamsnap

The features of this IGV-snapshot-automator are entirely limited to commands that can be used in IGV batch script;

• https://github.com/igvteam/igv/wiki/Batch-commands

• https://software.broadinstitute.org/software/igv/batch

• https://software.broadinstitute.org/software/igv/PortCommands

---

IGV Snapshot Automator

A script to automatically create and run IGV snapshot batchscripts. This script will first write an IGV batch script for the supplied input files, then load all supplied files for visualization (.bam, etc) in a headless IGV session and take snapshots at the locations defined in the regions.bed file.

Designed for use on Linux systems, and intended to be used as a component of sequencing analysis pipelines.

Usage

Download IGV

Use the included Makefile recipe to download a copy of IGV

make install

Run Snapshotter

• Put your chromosome regions to visualize in the regions.bed file (provided), or another BED format file

• Locate your files to visualize (e.g. .bam & .bam.bai files)

• Create and run batchscript. Example command:

$ python make_IGV_snapshots.py /path/to/alignments1.bam /path/to/alignments2.bam

Demo

To run the script on the included demo files:

$ python make_IGV_snapshots.py test_data/test_alignments.bam test_data/test_alignments2.bam

Options

See python make_IGV_snapshots.py --help for available options. Here are a few:

• -r: Path to the BED formatted regions file to use (defaults to the included demo regions.bed)
• -nosnap: Make batchscript without taking snapshots
• -g: Genome to use, e.g. hg19
• -ht: Height of the snapshot, default is 500
• -o: Name of the output directory to save the snapshots in (defaults to IGV_Snapshots)
• -bin: Path to the IGV jar binary to run (defaults to igv.jar)
• -mem: Memory to allocate to IGV (MB)
• -suffix: Filename suffix to place before '.png' in the snapshots
• -onlysnap: Skip batchscript creation and only run IGV using the supplied batchscript file
• -nf4: "Name field 4" mode, uses values saved in 4th field of the regions.bed file as the output filename of the PNG snapshot. Use this when you have pre-made filenames you wish to use for each snapshot.
• -s or -group_by_strand: Group alignment(s) by read strand with forward on top and reverse on the bottom.

Example Output

Notes

Default memory allotment is set at 4GB; this can be changed with the -mem argument (e.g. -mem 1000 sets memory to 1GB).

IGV may take several minutes to run, depending on the number of input files and regions to snapshot. Stdout messages from the program may not appear immediately in the console.

Containers

Docker and Singularity container files are included. Pre built container images can be found on Dockerhub at https://hub.docker.com/repository/docker/stevekm/igv-snapshot-automator

Docker

The Docker container can be built with the included Makefile recipe

make docker-build

The test data can be run with

make docker-test

Singularity

The Singularity container can be built using Docker with the included Makefile recipe

make singularity-build

The test data can be run with

make singularity-test

Software Requirements

• Python 2.7 or 3+
• bash version 4.1.2+
• IGV (download script provided in bin directory)
• Xvfb
• xdpyinfo
• Java runtime environment
• Docker or Singularity for building and running containers

Notes

Some alternative implementations of the same basic methodology used here for creating IGV snapshots can also be found implemented in Nextflow pipelines;

• https://github.com/stevekm/IGV-snapshot-nf

• https://github.com/NYU-Molecular-Pathology/NGS580-nf/blob/3ba2f970c3fbee56080ba60727f7bf43cb1be3b2/main.nf#L4701-L4876

Running a batch script on IGV:

• https://software.broadinstitute.org/software/igv/batch

• https://software.broadinstitute.org/software/igv/PortCommands

Details on interpretation of IGV visualizations can be found here:

• https://software.broadinstitute.org/software/igv/book/export/html/37

IGV available preferences which could be included in IGV batch scripts:

• https://software.broadinstitute.org/software/igv/prefs.properties

igv-reports which makes HTML report outputs with embedded Javascript IGV viewer

• https://github.com/igvteam/igv-reports

igv-snapshot-automator's Issues

Unable to change genome

The script (-g option or changing the source code directly) always default to the first genome in IGV. I couldn't make it to using the second genome which is hg38 on my machine.

IGV download URL needs an update

The new address is

http://data.broadinstitute.org/igv/projects/downloads/2.3/IGV_2.3.81.zip

Need to update containers to include the repo source code

Right now the containers do not seem to actually include the source code for the script, so add the script repo to the container

need support for BED file with extra columns

Right now, the input BED file can only have 2 or 3 columns. Need to adjust the script use input with extra columns.

invalid literal for int() with base 10 error

Hi,

First of all, thank you for this script-based IGV visualization.

When I run your demo dataset and also mine, I obtained error that I attached it as screenshot below:

Can I solve this issue?

Thanks for your interest.

Best wishes.

Load Bam files from URL

Hi,
I am trying to take a few screenshots of specific variants from a very large bam files on a google bucket. I am wondering if there is a way of loading bam files from URL in IGV-snapshot-automator?

I tried launching the igv.jar inside this folder and attempted to grant google access but it was blocked. I'm wondering if anybody has done this before and could you please help me troubleshoot it?

Thank you!

Snapshot on Pacbio hifi reads bam file

Dear author of IGV-snapshot-automator,

I'm trying to use pacbio long reads mapped bam file to generate IGV snapshot. It's doing ok with the raw reads aligned bam file. But generating exceptions on ccs reads aligned bam file with following messages:

ERROR [2021-08-30 12:03:43,561] [DefaultExceptionHandler.java:49] [AWT-EventQueue-0] Unhandled exception
java.lang.NullPointerException
at org.broad.igv.sam.AlignmentPacker.getGroupValue(AlignmentPacker.java:419)
at org.broad.igv.sam.AlignmentPacker.packAlignments(AlignmentPacker.java:88)
at org.broad.igv.sam.AlignmentInterval.packAlignments(AlignmentInterval.java:170)
at org.broad.igv.sam.AlignmentDataManager.packAlignments(AlignmentDataManager.java:282)
at org.broad.igv.sam.AlignmentDataManager.load(AlignmentDataManager.java:341)
at org.broad.igv.sam.CoverageTrack.load(CoverageTrack.java:184)
at org.broad.igv.ui.panel.DataPanel.lambda$load$3(DataPanel.java:225)
at org.broad.igv.ui.panel.DataPanel.load(DataPanel.java:229)
at org.broad.igv.ui.panel.DataPanel.paintComponent(DataPanel.java:156)
at javax.swing.JComponent.paint(JComponent.java:1056)
at javax.swing.JComponent.paintChildren(JComponent.java:889)
at org.broad.igv.ui.panel.DataPanelContainer.paintChildren(DataPanelContainer.java:149)
at javax.swing.JComponent.paint(JComponent.java:1065)
at javax.swing.JComponent.paintChildren(JComponent.java:889)
at javax.swing.JComponent.paint(JComponent.java:1065)
at javax.swing.JComponent.paintChildren(JComponent.java:889)
at javax.swing.JComponent.paint(JComponent.java:1065)
at javax.swing.JViewport.paint(JViewport.java:728)
at javax.swing.JComponent.paintChildren(JComponent.java:889)
at javax.swing.JComponent.paint(JComponent.java:1065)
at javax.swing.JComponent.paintChildren(JComponent.java:889)
at javax.swing.JComponent.paint(JComponent.java:1065)
at javax.swing.JComponent.paintChildren(JComponent.java:889)
at javax.swing.JComponent.paint(JComponent.java:1065)
at javax.swing.JComponent.paintChildren(JComponent.java:889)
at javax.swing.JComponent.paint(JComponent.java:1065)
at javax.swing.JComponent.paintChildren(JComponent.java:889)
at javax.swing.JComponent.paint(JComponent.java:1065)
at javax.swing.JLayeredPane.paint(JLayeredPane.java:586)
at javax.swing.JComponent.paintChildren(JComponent.java:889)
at javax.swing.JComponent.paintToOffscreen(JComponent.java:5217)
at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(RepaintManager.java:1579)
at javax.swing.RepaintManager$PaintManager.paint(RepaintManager.java:1502)
at javax.swing.BufferStrategyPaintManager.paint(BufferStrategyPaintManager.java:306)
at javax.swing.RepaintManager.paint(RepaintManager.java:1272)
at javax.swing.JComponent.paint(JComponent.java:1042)
at java.awt.GraphicsCallback$PaintCallback.run(GraphicsCallback.java:39)
at sun.awt.SunGraphicsCallback.runOneComponent(SunGraphicsCallback.java:79)
at sun.awt.SunGraphicsCallback.runComponents(SunGraphicsCallback.java:116)
at java.awt.Container.paint(Container.java:1978)
at java.awt.Window.paint(Window.java:3906)
at javax.swing.RepaintManager$4.run(RepaintManager.java:842)
at javax.swing.RepaintManager$4.run(RepaintManager.java:814)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:814)
at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:789)
at javax.swing.RepaintManager.prePaintDirtyRegions(RepaintManager.java:738)
at javax.swing.RepaintManager.access$1200(RepaintManager.java:64)
at javax.swing.RepaintManager$ProcessingRunnable.run(RepaintManager.java:1732)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
INFO [2021-08-30 12:03:46,091] [CommandExecutor.java:235] [pool-2-thread-2] OK
INFO [2021-08-30 12:04:01,946] [BatchRunner.java:72] [pool-2-thread-2] Executing Command: exit

Any idea of where goes wrong with the hifi(ccs) reads mapping file? The raw reads bam and hifi reads bam looks very close to each other except a couple of unique bam tag fields.

Thanks for helping!
Wen

The figure are small

Hi,

I find this is a really useful tool. However, when I run it on my server, the generated figures are very small.

Any reason for this? Thank you!

Genome should be loaded before loading BAM files

for file in input_files:
    append_string("load " + file, IGV_batchscript_file)
# add the genome version
append_string("genome " + genome_version, IGV_batchscript_file)

The "genome" command should go before "load", because once a new genome is loaded, we'll need to load the files again,

Add Docker container

Need a Dockerfile as well

an "hs_err_pid22398.log" produced

an "hs_err_pid22398.log" produced and the procedure is stopped. why and how to solve this problem.

the content in this files is below:

A fatal error has been detected by the Java Runtime Environment:

SIGSEGV (0xb) at pc=0x00007fb0d0181401, pid=22398, tid=22408

JRE version: OpenJDK Runtime Environment (11.0.9+11) (build 11.0.9+11-Ubuntu-0ubuntu1.18.04.1)

Java VM: OpenJDK 64-Bit Server VM (11.0.9+11-Ubuntu-0ubuntu1.18.04.1, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)

Problematic frame:

C [libjimage.so+0x2401]

Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %d %P %E" (or dumping to /software_users/chenliang/IGV-snapshot-automator/core.22398)

If you would like to submit a bug report, please visit:

https://bugs.launchpad.net/ubuntu/+source/openjdk-lts

--------------- S U M M A R Y ------------

Command Line: -Xmx4000m bin/IGV_2.3.81/igv.jar -b IGV_Snapshots/IGV_snapshots.bat

Host:

Possible to sort by base?

Trying to run on a Mac

Hi,

This is absolutely amazing thank you guys for making this! I am trying to run it on my MacOS and getting stuck when it reaches the IGV command:

Running the IGV command...
/bin/sh: xvfb-run: command not found

Do you know if there is some alternative way to run the IGV command?

Thank you so much!

How to input the region in region.bed file if it lies between 2 chromosomes?

Hi, Thank you for your work on the automator, it is really helpful!
While giving the regions that we want to focus on in the regions.bed file, how can we give regions that fall between 2 chromosomes; lets say from 5794234 in chr 3 to 20910230 in chr 8?

Softclipped regions

Hey,

already super helpful tool!
Is there a possibility to somehow make soft-clipped bases visible for the snapshot?

Thanks in advance!

HI

i install xvfb but it shown

$ ./bin/Xvfb :0 -nolisten tcp -fp lib/X11/fonts/misc
(WW) Could not open RGB file "/usr/X11R6/lib/X11/rgb.txt"; will use built-in copy.
error opening security policy file /usr/X11R6/lib/X11/xserver/SecurityPolicy
_FontTransOpen: Unable to Parse address lib/X11/fonts/misc
Could not init font path element lib/X11/fonts/misc, removing from list!

Fatal server error:
could not open default font 'fixed'

How to add the center line in the middle of the picture in the test

hello，please In test_data dirs the parameter of the center line in the middle of the image generated in data? I tried some but failed，in IGV_snapshots.bat file which parameter works？

$ cat IGV_snapshots.bat
new
genome hg19
snapshotDirectory IGV_Snapshots
load test_data/test_alignments.bam
load test_data/test_alignments2.bam
maxPanelHeight 500
goto chr1:713167-714758
snapshot chr1_713167_714758_h500.png
goto chr1:713500-714900
snapshot chr1_713500_714900_h500.png
goto chr1:714000-715000
snapshot chr1_714000_715000_h500.png
goto chr1:714001-714001
snapshot chr1_714001_714001_h500.png
exit

need to check for bai's

IGV will not load BAM's without BAI files, but does not return an error, would save some headache if the script checked for BAI with BAM inputs

Only output IGV_snapshots.bat without *png

Hi,
I meet the error when I run IGV-snapshot with comand: python IGV-snapshot-automator-master/make_IGV_snapshots.py -r test.bed -o igv -g ./genome.fa -mem 64000 -s test.bam. There were no error. My bam file size is 80GB.
But when I just extract bam files with positon in test.bed. It works and output *png. Why cause this? Thanke you.

count details of the region/locus of interest

hello. this automation is wonderful. could you add a line of code to show on the saved .png or output the count details of the region/locus of interest? Most of the time, all we need are in that window (strand bias, alt reads, ref reads, indel reads). Thanks.

Fix the default path to IGV .jar file

Right now the default path for the -bin arg is set to bin/IGV_2.3.81/igv.jar, this should be changed to something like bin/igv.jar so its version-agnostic and you can just symlink in the bin dir the path to the jar if you want

color the positive and negative strand

Hello.I‘m using this software and it helps me a lot.
I was wondering if it could color the positive and negative strand .

Thank you.

Instructions to get this working in cluster without X11

Figured I would share this here as this might help someone down the line. I use IGV in remote cluster where X11 is not available. I resolved this by using the instructions here - http://thedusseldorfer.blogspot.com/2013/09/remotely-plotting-with-igv-even-without.html. Note that this does not require sudo privileges.

Steps in brief:

1. Download Xvfb.tgz and Xfnts.tgz from http://ftp.xfree86.org/pub/XFree86/4.8.0/binaries/
2. Decompress them
3. Run this ./Xvfb :0 -nolisten tcp -fp /path/to/lib/X11/fonts/misc &
4. Set export DISPLAY=:0
5. Now run IGV batch file: /path/to/igv.sh --batch /path/to/batch/file

I create IGV batch file using IGV-snapshot-automator, and with these batch files, I then create snapshots as described above. Hope this helps! If there is a better way to do this, or to get above Xvfb to work with xvfb-run and xdpyinfo without sudo privileges, please let me know.

testing run is not working

(base) shiyi@ip-192-168-1-231:~/mnt/atac-fetal-2021-11/IGV-snapshot-automator$ python make_IGV_snapshots.py test_data/test_alignments.bam test_data/test_alignments2.bam

Reference genome:
hg19

Track height:
500

IGV binary file:
/home/shiyi/mnt/atac-fetal-2021-11/IGV-snapshot-automator/igv.jar

Output directory will be:
/home/shiyi/mnt/atac-fetal-2021-11/IGV-snapshot-automator/IGV_Snapshots

Batchscript file will be:
/home/shiyi/mnt/atac-fetal-2021-11/IGV-snapshot-automator/IGV_Snapshots/IGV_snapshots.bat

Region file:
/home/shiyi/mnt/atac-fetal-2021-11/IGV-snapshot-automator/regions.bed

Input files to snapshot:

test_data/test_alignments.bam
test_data/test_alignments2.bam

Making the output directory...

Writing IGV batch script to file:
/home/shiyi/mnt/atac-fetal-2021-11/IGV-snapshot-automator/IGV_Snapshots/IGV_snapshots.bat


Getting regions from BED file...

Read 4 regions
xdpyinfo:  unable to open display ":1".

Open Xvfb port found on:
1


IGV command is:
xvfb-run --auto-servernum --server-num=1 java -Xmx4000m -jar /home/shiyi/mnt/atac-fetal-2021-11/IGV-snapshot-automator/igv.jar -b /home/shiyi/mnt/atac-fetal-2021-11/IGV-snapshot-automator/IGV_Snapshots/IGV_snapshots.bat


Current time is:
2022-01-10 18:43:33.046202


Running the IGV command...
/bin/sh: 1: xvfb-run: not found
b''

IGV finished; elapsed time is:
0:00:00.001256

(base) shiyi@ip-192-168-1-231:~/mnt/atac-fetal-2021-11/IGV-snapshot-automator$ 





i am not seeing any snapshot saved in the directory, do you know what can be the problem?
thank you

Different outputs

Hi,

I use the IGV-snapshot-automator with "xvfb-run" command like this;

xvfb-run java -jar bin/IGV_2.3.81/igv.jar -b IGV_Snapshots/IGV_snapshots.bat

However, I get different output PNGs whenever I run the command with the same .bat file. Any idea? Thanks.

.Genome file input

Is there a way to make the .genome file headlessly on the command line? I made a .chrom.sizes file which takes the place of the .genome file but the alignments with annotations would be much preferred.

Thank you.

Jeff

showing number of reads supporting each allele

Hi there
I want to use your tool to draw diagrams for a list of variants (one each), which I am supplying in a bed file. Is it possible to tweak your script / batch script to get the snapshots to show the number of reads carrying each allele at the specified position? I can get this in IGV when I hover over the coverage for that site ..
I would appreciate any ideas you might have, thanks!

replace xvfb on Mac

Hi,

I am having problem installing xvfb on mac. Is there any way to replace the xvfb to other alternative in the code?