imagoxv / nanoasv Goto Github PK
View Code? Open in Web Editor NEWNanoASV official repo
License: GNU General Public License v3.0
NanoASV official repo
License: GNU General Public License v3.0
I need to add a checkpoint system that could allow to resume data analysis in case of error to avoid re-computing
Same, I should add a R_ONLY option to phyloseq data in case the metadata.csv file was not working
for example:
which mafft > /dev/null || \
{ echo "error message" ; exit 1 ; }
Tests should occur early in the script; before any time-consuming computation.
Need to singleline SILVA, directly in dockerfile because, it's the moment.
need to extract reference seq to build tree with fastTree.
Need to feed Rscript with the tree and then inject it in phyloseq object.
Almost there
title
Porechop uses as many cpu as possible no matter the parallelization I wrote.
It might detect and use whatever is asked. It might even do that in parallel somehow
It seems a pretty lightweight computation still.
Chimera detection seems long for some highly sequenced barcodes.
I need to add a subsampling step before chimera detection, maybe something like --subsampling XX * 2 to allow for some buffer sequences.
Chimera detection seems long for some highly sequenced barcodes.
I need to add a subsampling step before chimera detection, maybe something like --subsampling XX * 2 to allow for some buffer sequences.
For a given repository, GitHub can parse and expose a recommended way to cite the repository. It only requires a CITATION.cff
file. Check for instance:
https://github.com/frederic-mahe/mumu/blob/main/CITATION.cff
which is exposed as:
Mahé, F. (2023). mumu: post-clustering curation tool for metabarcoding data (Version 1.0.2) [Computer software]. https://github.com/frederic-mahe/mumu
First test on real dataset had memory handling issues for some barcodes.
Dataset has to be subsampled
Numerical taxonomic richness seemed to have increased comparing to my previous treatment. I have to investigate.
I cannot make it run pn MK1C because chopper was compiled for amd64
Need to find a way
Hi @frederic-mahe, I tried to make a release to see how it works. However, my binary is too big (~5Gb). Max allowed is 2Gb.
Any idea on how to overcome this ?
Arthur
Need to specify the versions in the docker file installation so it's always the same tool versions used
I should probably fix the library verisons as well
I used bwa for simplicity sake, but now I need to change it for bwa-mem2 which is supposed to be more memory efficient and faster
Phyloseq installation is way too long.
Maybe because of depencies = TRUE
I wonder if possible to parallelize the indexing step (which is clearly the most computer intensive during the build process
Singletons are discarded anyway, it will reduce computation time
This drives me crazy.
On the IFB cluster, NanoASV running with singularity :
/shared/software/modules/4.6.1/init/bash: line 37: /usr/bin/tclsh: No such file or directory
The whole purpose of a container is to NOT LOOK OUTSIDE OF IT isn't it ?
Vsearch seems to never detect chimera with default parameters.
I think it lies on the fact that sequences are not dereplicated and therefore do not have a "count" section in fasta header.
However, I think dereplication might not work because vsearch expects 100% similarity. Which is rarely (if not) achieved with nanopore amplicon sequencing.
Efficient dereplication would come from accepting a certain variability threshold that would end up being clustering. Such clustering with vsearch performs well with a --id 0.7. Which is significantly lower than what we would want to accept for dereplication. If clustering, then it's not ASV treatment anymore.
I need to discuss it with you @frederic-mahe
For unknown reasons, the unknown cluster abundance < 5 are not discarded anymore and they make my alignement a nightmare with all these singletons. I got to change that.
WARNING: could not mount /etc/localtime: not a directory
This warning message does not appear when running with docker but does with singularity.
It seemingly has no consequences on downstream analyses.
Error: package or namespace load failed for ‘phyloseq’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/home/imago/R/x86_64-pc-linux-gnu-library/4.3/stringi/libs/stringi.so':
libicui18n.so.70: cannot open shared object file: No such file or directory
Execution halted
Something might have changed.
I'm pretty sure that's because of ubuntu:latest
Gotta change for ubuntu 22.04
I'm sure at some point it was specified. IDK what happened
conda install -c bioconda chopper
The problem is that I still need to install conda in my dockerfile
Line 428 in 79f3132
with the option --fasta_width 0
, vsearch produces unwrapped/unfolded fasta files.
It seems that singularity is instantly called when running nanoasv, which makes the following
singularity run nanoasv --options
uneccessary. If nanoasv singularity file is executable, then just ./nanoasv
or nanoasv
is you put it in /opt/ and add it to the $PATH
A nice way to do it
echo 'export PATH=$PATH:/opt/' >> ~/.bashrc && source ~/.bashrc
which makes
~$ nanoasv
WARNING: could not mount /etc/localtime: not a directory
______________________________________
/ Error: -d needs an argument, I don't \
\ know where your sequences are. /
--------------------------------------
\ ^__^
\ (xx)\_______
(__)\ )\/\
U ||----w |
|| ||
Lovely
Step 4/9 : Adapter trimming with Porechop
Step 5/9 : Subsampling
Step 6/9 : Reads alignements with bwa against SILVA_138.1
environment: line 1: 218 Segmentation fault (core dumped) bwa mem ${DB}/SILVA_IDX "${FILE}" 2> /dev/null > "${FILE}.sam"
environment: line 1: 221 Segmentation fault (core dumped) bwa mem ${DB}/SILVA_IDX "${FILE}" 2> /dev/null > "${FILE}.sam"
Step 7/9 : Skipped - no unknown sequence
Step 8/9 : Phylogeny with MAFFT and FastTree
Step 9/9 : Phylosequization with R and phyloseq
Data treatment is over.
NanoASV took 144 seconds to perform.
This indicates a memory related error.
If only the MK1C was running dozens of useless job in background.
I'll find a way
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.