Combining multiple Ion chip runs about amptk

[nextgenusfs]

Great, happy to hear that it is useful!

Yes, you can combine the runs via concatenating the demuxed files - but you need to make sure that each sample has a unique label. So the function of the --mult_samples is to add a "base name" to each read so that it is distinct from another run, i.e.

amptk ion -i run1.fastq.gz -o run1 --mult_samples r1 --barcode_fasta labels.fa

This will create fastq headers that have 'r1.' at the beginning of the sample name, i.e.

$ amptk ion -i ion.test.fastq -o run1 --mult_samples r1
-------------------------------------------------------
[Apr 04 03:04 PM]: OS: MacOSX 10.13.3, 8 cores, ~ 17 GB RAM. Python: 2.7.11
[Apr 04 03:04 PM]: AMPtk v1.1.2-c1661a0, USEARCH v9.2.64, VSEARCH v2.7.0
[Apr 04 03:04 PM]: Foward primer: AGTGARTCATCGAATCTTTG,  Rev comp'd rev primer: GCATATCAATAAGCGGAGGA
[Apr 04 03:04 PM]: Loading FASTQ Records
[Apr 04 03:04 PM]: 2,000 reads (1.6 MB)
-------------------------------------------------------
[Apr 04 03:04 PM]: Concatenating Demuxed Files
[Apr 04 03:04 PM]: 2,000 total reads
[Apr 04 03:04 PM]: 1,409 valid Barcode
[Apr 04 03:04 PM]: 1,406 Fwd Primer found, 1,151 Rev Primer found
[Apr 04 03:04 PM]: 34 discarded too short (< 100 bp)
[Apr 04 03:04 PM]: 1,372 valid output reads
[Apr 04 03:04 PM]: Found 19 barcoded samples
                Sample:  Count
              r1.BC.27:  95
              r1.BC.23:  90
              r1.BC.17:  90
              r1.BC.28:  88
              r1.BC.20:  82
              r1.BC.73:  80
              r1.BC.18:  77
              r1.BC.16:  72
              r1.BC.15:  72
              r1.BC.21:  72
              r1.BC.10:  71
              r1.BC.22:  68
              r1.BC.11:  66
              r1.BC.14:  65
               r1.BC.9:  65
              r1.BC.24:  62
              r1.BC.12:  60
              r1.BC.19:  53
               r1.BC.5:  44
[Apr 04 03:04 PM]: Output file:  run1.demux.fq.gz (242.0 KB)
[Apr 04 03:04 PM]: Mapping file: run1.mapping_file.txt

The other way is to just provide unique labels in the barcode fasta file:

>sample1
CTAAGGTAAC
>sample2
TAAGGAGAAC
>sample3
AAGAGGATTC
>sample4
TACCAAGATC
...

And then finally you can specify unique names in a mapping file as well, i.e.

#SampleID	BarcodeSequence	LinkerPrimerSequence	ReversePrimer	phinchID	DemuxReads	Treatment
sample1	CAGAAGGAAC	CCATCTCATCCCTGCGTGTCTCCGACTCAGCAGAAGGAACAGTGARTCATCGAATCTTTG	TCCTCCGCTTATTGATATGC	r1.BC.5	44	no_data
sample2	TGAGCGGAAC	CCATCTCATCCCTGCGTGTCTCCGACTCAGTGAGCGGAACAGTGARTCATCGAATCTTTG	TCCTCCGCTTATTGATATGC	r1.BC.9	65	no_data
sample3	CTGACCGAAC	CCATCTCATCCCTGCGTGTCTCCGACTCAGCTGACCGAACAGTGARTCATCGAATCTTTG	TCCTCCGCTTATTGATATGC	r1.BC.10	71	no_data
sample4	TCCTCGAATC	CCATCTCATCCCTGCGTGTCTCCGACTCAGTCCTCGAATCAGTGARTCATCGAATCTTTG	TCCTCCGCTTATTGATATGC	r1.BC.11	66	no_data

And then use that mapping file to demux

$ amptk ion -i ion.test.fastq -o run1 -m my_mappingfile.txt
-------------------------------------------------------
[Apr 04 03:09 PM]: OS: MacOSX 10.13.3, 8 cores, ~ 17 GB RAM. Python: 2.7.11
[Apr 04 03:09 PM]: AMPtk v1.1.2-c1661a0, USEARCH v9.2.64, VSEARCH v2.7.0
[Apr 04 03:09 PM]: Foward primer: AGTGARTCATCGAATCTTTG,  Rev comp'd rev primer: GCATATCAATAAGCGGAGGA
[Apr 04 03:09 PM]: Loading FASTQ Records
[Apr 04 03:09 PM]: 2,000 reads (1.6 MB)
-------------------------------------------------------
[Apr 04 03:09 PM]: Concatenating Demuxed Files
[Apr 04 03:09 PM]: 2,000 total reads
[Apr 04 03:09 PM]: 248 valid Barcode
[Apr 04 03:09 PM]: 248 Fwd Primer found, 210 Rev Primer found
[Apr 04 03:09 PM]: 2 discarded too short (< 100 bp)
[Apr 04 03:09 PM]: 246 valid output reads
[Apr 04 03:09 PM]: Found 4 barcoded samples
                Sample:  Count
               sample3:  71
               sample4:  66
               sample2:  65
               sample1:  44
[Apr 04 03:09 PM]: Output file:  run1.demux.fq.gz (42.2 KB)
[Apr 04 03:09 PM]: Mapping file: my_mappingfile.txt

If you already have your demuxed runs and don't want to redo it, you could also do a find/replace with sed on your demuxed runs:

sed 's/barcodelabel=/barcodelabel=run1./g' demux.fq > demux.fixed.fq

And then finally you can just concatenate using cat which will work even on gzipped files:

cat run1.demux.fq.gz run2.demux.fq.gz > combined.demux.fq.gz

[JonathanVanHamme]

Brilliant, thanks for the quick and thorough reply, Jon! It is working like a charm.

[nextgenusfs]

Thanks Jon! Is the data quality improved on the Ion S5 XL? We are still running the PGM - no plans to upgrade at the moment, but just curious. And let me know if you run into any more problems with AMPtk or have some features that you'd like to see incorporated.

[JonathanVanHamme]

Hey Jon, I honestly haven’t done a thorough comparison of data quality, I’m going to say it is similar. The real big benefit to the upgrade is when you buy the Ion Chef as well, it really streamlines processing and saves a ton of hands on time - the Chef is also better at chip loading. It feels like cheating it is so easy. The throughput is much higher as well with only a slight increase in operating costs. I mostly run 400 bp chemistry and, with two runs a day, you can pump out 15 Gb in a day (the XL server helps cut down on processing time; they have 600 bp chemistry, but I haven’t played with it yet for the S5). I think the new 550 chip that came out has 200 million wells, but is limited to 200 bp because of the small size of the wells. I have attached a run report from the first metagenome we sequenced. Let me know if you want any data to play around with and I can send it your way. Right now we are doing bacterial genomes, metagenomes, chloroplast genomes, 16S, ITS, CO1 (insects, fish). If you want me to run some test samples for you, I’m sure we could fit them in! I’ll definitely put some thought into other features - we were attracted to it because of the versatility you put in for the different targets, and while I haven’t spent too much time with it, QIIME2 irritated me with all of their special file formats. Have a great weekend! Jon Jonathan Van Hamme, PhD Associate Professor, Microbiology http://faculty.tru.ca/jvanhamme/ Director, TRUGen Applied Genomics Laboratory<https://www.tru.ca/trugen.html> [cid:[email protected]] Member, Biological and Chemical Research Defence Committee<http://bcdrc.ca/> Managing Editor, Biodegradation<https://link.springer.com/journal/10532> On Apr 6, 2018, at 10:18 AM, Jon Palmer <[email protected]<mailto:[email protected]>> wrote: Thanks Jon! Is the data quality improved on the Ion S5 XL? We are still running the PGM - no plans to upgrade at the moment, but just curious. And let me know if you run into any more problems with AMPtk or have some features that you'd like to see incorporated. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub<#38 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AKE3Jq4cXAm7iKLNMd4d6VAslq4gdxTYks5tl6NKgaJpZM4THTWs>.