fei38750 / mtd Goto Github PK
View Code? Open in Web Editor NEWMTD: a unique pipeline for host and meta-transcriptome joint and integrative analyses of RNA-seq data
License: Other
mtd's People
Contributors
Stargazers
Watchers
mtd's Issues
Paired samples
Dear Fei,
For the differential expression analysis, is it possible to specify somewhere that the samples being compared are paired?
Best regards,
Leonor
Installation failing
Cloned the git as indicated and went on to the installation:
bash ./MTD/Install.sh -t 60 -p ./miniconda
and the install throws error as shown in pictures below.
Note that the clone and the miniconda are downloaded/installed in a mounted data storage where everything else is installed. I already have Kraken2 installed as well as other programs in there. 7.8To of free storage. 96 cores and 400 Go Ram Fedora Instance.
What is the issue here ?
The installations is stuck when building a new human database
Dear authors,
I've been trying to install the MTD software for a few times but I'm always encountering the same issue which is that when creating the new human db the installations just stays trying to do so but it doesn't move on. The expected file with the human database is emtpy all the time. Do you have any idea on how to fix it or how I could figure out what's the problem behind this?
Here I share with you the latest messages provided by the installation:
Building a new DB, current time: 02/18/2023 17:22:40
New DB name: /path/MTD/human_blastdb/human_blastdb
New DB title: /path/MTD/hisat2_index_human/genome.fa
Sequence type: Nucleotide
Deleted existing Nucleotide BLAST database named /path/MTD/human_blastdb/human_blastdb
Keep MBits: T
Maximum file size: 1000000000B
Best regards,
Leonor
Pipeline running but barely any output
running into similar issues where many folder coming out of the pipeline are completely empty (see screenshot below)
as well, it keep throwing me a bunch of errors towards the end (see second screenshot) and I wouldn't be surprised if there was some other issues earlier, but my tmux doesn't let me go up any further for some reasons.
Is there any log file created by the code that could help me track where these issues originate and help me understand what's going on here ?
Multiple installation error prompts
Dear author,
thank you very much for providing this useful pipeline. I have completed the MTD installation process. While reviewing the installation logs, I noticed multiple error prompts:
error prompts1：
unpigz error
MTD installation progress: >>>[15%]
downloading virome database...
--2023-06-16 08:45:33-- https://www.genome.jp/ftp/db/virushostdb/virushostdb.genomic.fna.gz
Resolving www.genome.jp (www.genome.jp)... 133.103.200.20
Connecting to www.genome.jp (www.genome.jp)|133.103.200.20|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘virushostdb.genomic.fna.gz’
..........................
173300K .......................... 1.89M=4m16s
2023-06-16 08:49:50 (676 KB/s) - ‘virushostdb.genomic.fna.gz’ saved [177486944]
unpigz: abort: write error on virushostdb.genomic.fna (Inappropriate ioctl for device)
error corresponding to line 52-58 in install.sh
echo 'MTD installation progress:'
echo '>>>[15%]'
echo 'downloading virome database...'
conda activate MTD
wget -c https://www.genome.jp/ftp/db/virushostdb/virushostdb.genomic.fna.gz
unpigz virushostdb.genomic.fna.gz
cat Installation/M33262_SIVMM239.fa virushostdb.genomic.fna > viruses4kraken.fa
error prompts2：
rsync error
MTD installation progress:
>>>>[20%]
Preparing microbiome (virus, bacteria, archaea, protozoa, fungi, plasmid, UniVec_Core) database...
Step 1/2: Performing rsync file transfer of requested files
Rsync file transfer complete.
Step 2/2: Assigning taxonomic IDs to sequences
All files processed, cleaning up extra sequence files... done, library complete.
Masking low-complexity regions of downloaded library... done.
Step 1/2: Performing rsync file transfer of requested files
Rsync file transfer complete.
Step 2/2: Assigning taxonomic IDs to sequences
All files processed, cleaning up extra sequence files... done, library complete.
Masking low-complexity regions of downloaded library... done.
Step 1/2: Performing rsync file transfer of requested files
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(701) [generator=3.2.3]
rsync error: received SIGUSR1 (code 19) at main.c(1592) [receiver=3.2.3]
rsync_from_ncbi.pl: rsync error, exiting: 5120
Step 1/2: Performing rsync file transfer of requested files
Rsync file transfer complete.
Step 2/2: Assigning taxonomic IDs to sequences
All files processed, cleaning up extra sequence files... done, library complete.
Masking low-complexity regions of downloaded library... done.
Downloading plasmid files from FTP...xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
error corresponding to line 66-79 in install.sh
echo 'MTD installation progress:'
echo '>>>>[20%]'
echo 'Preparing microbiome (virus, bacteria, archaea, protozoa, fungi, plasmid, UniVec_Core) database...'
# Kraken2 database building - Microbiome
DBNAME=kraken2DB_micro
kraken2-build --download-taxonomy --threads $threads --db $DBNAME $kmer $min_l $min_s
kraken2-build --download-library archaea --threads $threads --db $DBNAME $kmer $min_l $min_s
kraken2-build --download-library bacteria --threads $threads --db $DBNAME $kmer $min_l $min_s
kraken2-build --download-library protozoa --threads $threads --db $DBNAME $kmer $min_l $min_s
kraken2-build --download-library fungi --threads $threads --db $DBNAME $kmer $min_l $min_s
kraken2-build --download-library plasmid --threads $threads --db $DBNAME $kmer $min_l $min_s
kraken2-build --download-library UniVec_Core --threads $threads --db $DBNAME $kmer $min_l $min_s
kraken2-build --add-to-library viruses4kraken.fa --threads $threads --db $DBNAME $kmer $min_l $min_s
kraken2-build --build --threads $threads --db $DBNAME $kmer $min_l $min_s
error prompts3：
rsync error
Downloading nucleotide gb accession to taxon map... done.
Downloading nucleotide wgs accession to taxon map... done.
Downloaded accession to taxon map(s)
Downloading taxonomy tree data...rsync: read errors mapping "/taxonomy/taxdump.tar.gz" (in pub): Stale file handle (116)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1816) [generator=3.2.3]
Masking low-complexity regions of new file... done.
Added "kraken2DB_rhesus/GCF_003339765.1_Mmul_10_genomic.fa" to library (kraken2DB_rhesus)
Creating sequence ID to taxonomy ID map (step 1)...
Accession to taxid map files are required to build this DB.
Run 'kraken2-build --db kraken2DB_rhesus --download-taxonomy' again?
error corresponding to line 105-118 in install.sh
echo 'MTD installation progress:'
echo '>>>>>>>>[40%]'
echo 'Preparing host (rhesus monkey) database...'
# Kraken2 database building - Rhesus macaque
DBNAME=kraken2DB_rhesus
mkdir -p $DBNAME
cd $DBNAME
wget -c https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/003/339/765/GCF_003339765.1_Mmul_10/GCF_003339765.1_Mmul_10_genomic.fna.gz
unpigz GCF_003339765.1_Mmul_10_genomic.fna.gz
mv GCF_003339765.1_Mmul_10_genomic.fna GCF_003339765.1_Mmul_10_genomic.fa
cd ..
kraken2-build --download-taxonomy --threads $threads --db $DBNAME $kmer $min_l $min_s
kraken2-build --add-to-library $DBNAME/GCF_003339765.1_Mmul_10_genomic.fa --threads $threads --db $DBNAME $kmer $min_l $min_s
kraken2-build --build --threads $threads --db $DBNAME $kmer $min_l $min_s
If I only run the code in this specific area to eliminate the impact of the error prompts, instead of rerunning install.sh again , will it affect the use of MTD? Looking forward to your reply.
Best regards!
chan
Multiple errors running pipeline
Hello,
Thank you for making this pipeline available. I was able to install and start a run on my data (just using 2 paired-end samples at first, please let me know if the pipeline requires more than 2 samples). I have run into a few errors but have been able to fix them with simple syntax changes. However, I am now running into two errors that I'm not sure how to fix.
The first is during the "#DEG & Annotation...":
Still, it is able to halt this and keep running. The second error is then a "file not found" error so I'm assuming something upstream didn't work.
I do not have a computer science background but my colleagues in CS have also been unable to debug. They told me they ran into similar errors using the test run files as well. I appreciate any input!
Thanks,
Gillian
Pipeline finishing with no results
Like the title says - I ran the pipeline overnight and when I opened this morning it said it was done/didn't report any errors. However, there was nothing in my output folder except for the temp folder?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.