bowhan / pipipes Goto Github PK

View Code? Open in Web Editor NEW

55.0 9.0 39.0 609.86 MB

piRNA pipeline collection developed in the Zamore Lab and ZLab in UMass Med School

License: GNU General Public License v3.0

Shell 26.13% R 4.74% Perl 25.52% Python 2.43% C++ 6.79% C 0.16% HTML 33.78% CMake 0.32% JavaScript 0.11% Makefile 0.01%

pipipes's People

Contributors

Stargazers

Watchers

pipipes's Issues

Error while installing 'feather'

Hello, I am trying to install package 'rio' and get the following error while installing dependency 'feather'

installing source package 'feather' ...
** package 'feather' successfully unpacked and MD5 sums checked
** libs

*** arch - i386
Warning: running command 'make -f "Makevars" -f "C:/PROGRA~~1/R/R-devel/etc/i386/Makeconf" -f "C:/PROGRA~~1/R/R-devel/share/make/winshlib.mk" CXX='$(CXX1X) $(CXX1XSTD)' CXXFLAGS='$(CXX1XFLAGS)' CXXPICFLAGS='$(CXX1XPICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX1XLDFLAGS)' SHLIB_LD='$(SHLIB_CXX1XLD)' SHLIB="feather.dll" OBJECTS="RcppExports.o feather-read.o feather-types.o feather-write.o"' had status 127
ERROR: compilation failed for package 'feather'

removing 'C:/Users/User_0126_05/Documents/R/win-library/3.4/feather'
Warning in install.packages :
running command '"C:/PROGRA~~1/R/R-devel/bin/x64/R" CMD INSTALL -l "C:\Users\User_0126_05\Documents\R\win-library\3.4" C:\Users\USER_0~~1\AppData\Local\Temp\RtmpmSxgP5/downloaded_packages/feather_0.0.1.tar.gz' had status 1
Warning in install.packages :
installation of package ‘feather’ had non-zero exit status
ERROR: dependency 'feather' is not available for package 'rio'

removing 'C:/Users/User_0126_05/Documents/R/win-library/3.4/rio'
Warning in install.packages :
running command '"C:/PROGRA~~1/R/R-devel/bin/x64/R" CMD INSTALL -l "C:\Users\User_0126_05\Documents\R\win-library\3.4" C:\Users\USER_0~~1\AppData\Local\Temp\RtmpmSxgP5/downloaded_packages/rio_0.4.6.tar.gz' had status 1
Warning in install.packages :
installation of package ‘rio’ had non-zero exit status

When I try to install 'feather' separately, I still get the following error:

installing source package 'feather' ...
** package 'feather' successfully unpacked and MD5 sums checked
** libs

*** arch - i386
Warning: running command 'make -f "Makevars" -f "C:/PROGRA~~1/R/R-devel/etc/i386/Makeconf" -f "C:/PROGRA~~1/R/R-devel/share/make/winshlib.mk" CXX='$(CXX1X) $(CXX1XSTD)' CXXFLAGS='$(CXX1XFLAGS)' CXXPICFLAGS='$(CXX1XPICFLAGS)' SHLIB_LDFLAGS='$(SHLIB_CXX1XLDFLAGS)' SHLIB_LD='$(SHLIB_CXX1XLD)' SHLIB="feather.dll" OBJECTS="RcppExports.o feather-read.o feather-types.o feather-write.o"' had status 127
ERROR: compilation failed for package 'feather'

removing 'C:/Users/User_0126_05/Documents/R/win-library/3.4/feather'
Warning in install.packages :
running command '"C:/PROGRA~~1/R/R-devel/bin/x64/R" CMD INSTALL -l "C:\Users\User_0126_05\Documents\R\win-library\3.4" C:\Users\USER_0~~1\AppData\Local\Temp\RtmpmSxgP5/downloaded_packages/feather_0.0.1.tar.gz' had status 1
Warning in install.packages :
installation of package ‘feather’ had non-zero exit status

Can anybody help?

-Sam

TRANSGENE_MAPPING_DIR not set in piPipes_GenomeSeq.sh

Dear Bo Han,

I tried to run the piPipes dna pipeline and noticed the following in the output:

[bam_parse_region] fail to determine the sequence name.
[main_samview] region "chrXHet:-29-971" specifies an unknown reference name. Continue anyway.
[2015-09-17 11:49:00 CEST] Mapping transgene insertion by TEMP
piPipes_GenomeSeq.sh: line 291: /transgene.genome.bed: Permission denied
[samopen] SAM header is present: 15 sequences.
(ERR): bowtie2-align exited with value 141

Looking at piPipes_GenomeSeq.sh, it seems that the "Permission denied" error is due to the fact that line 109 (of current version) is commented:

# TRANSGENE_MAPPING_DIR=transgene_mapping && mkdir -p $TRANSGENE_MAPPING_DIR

This variable not being defined make it impossible for bedtools_piPipes bamtobed (line 291) to write its output: $TRANSGENE_MAPPING_DIR/transgene.genome.bed is just /transgene.genome.bed, and consequently not writeable.

Bug in piPipes_bed2_to_bedGraph

Hi Bo,

It seems that this utility is handling the very first position in each contig in a weird way: the signals for the first positions are not set to 0 when a new contig is encountered. Below is an example:

$ cat small.ChromInfo.txt
NW_004581714.1	2150616
NW_004582652.1	6536

$ cat small.bed2
NW_004581714.1	0	1	1000	1	+	TCGCATGATCTTGGTGGCCAATT
NW_004582652.1	0	1	10	1	+	TTACGCAGAAAGCATTGTCTAGAT

$ ~/data/piPipes/bin/piPipes_bed2_to_bedGraph --input small.bed2 -c small.ChromInfo.txt -o small.output.
$ cat small.output.Watson.bedGraph 
NW_004581714.1	0	1	1000.00000
NW_004582652.1	0	1	1010.00000

The correct result should be

NW_004581714.1	0	1	1000.00000
NW_004582652.1	0	1	10.00000

There might be a bug in the bed2_bedgraph_converter function. This bug will not cause serious problems in relatively complete genome assemblies, such as dm3, and mm10, as they contain just a few contigs (chromosomes). However, this will cause some serious problems for fragmented genome assemblies, such as the silkworm genome assembly from NCBI.

Thanks!
Yu

piPipes_smallRNA_bed2_to_bw.sh: line 32: 73659 Segmentation fault

Recently, I used piPipes step by step using data in manual, I ran the small RNA-Seq part:
piPipes small -i Zamore.SRA.ago3_het.ox.ovary.trimmed.fastq.gz -g dm3 -o Zamore.SRA.ago3_het.ox.ovary.piPipes_out 1> Zamore.SRA.ago3_het.ox.ovary.piPipes.stdout 2> Zamore.SRA.ago3_het.ox.ovary.piPipes.stderr
But it showed errors:
cat Zamore.SRA.ago3_het.ox.ovary.piPipes.stderr
Rscript execution error: No such file or directory
/mnt/isilon/wang_lab/caiyou/piPipes/bin/piPipes_smallRNA_bed2_to_bw.sh: line 32: 73659 Segmentation fault (core dumped) piPipes_bed2_to_bedGraph -i $OUTDIR/${INPUT_BED_NAME%bed2}sorted.bed2 -c $CHROM -p $CPU -o $OUTDIR/${INPUT_BED_NAME%bed2}sorted.
/mnt/isilon/wang_lab/caiyou/piPipes/bin/piPipes_smallRNA_bed2_to_bw.sh: line 33: 73663 Segmentation fault (core dumped) piPipes_bed2_to_bedGraph -i $OUTDIR/${INPUT_BED_NAME%bed2}sorted.uniq.bed2 -c $CHROM -p $CPU -o $OUTDIR/${INPUT_BED_NAME%bed2}sorted.uniq.
[bam_header_read] EOF marker is absent. The input is probably truncated.
Rscript execution error: No such file or directory
[bam_header_read] EOF marker is absent. The input is probably truncated.
Rscript execution error: No such file or directory
[bam_header_read] EOF marker is absent. The input is probably truncated.
Rscript execution error: No such file or directory
GPL Ghostscript 9.07: Unrecoverable error, exit code 1

The files in /common/dm3/
Bowtie2Index
BowtieIndex
Brennecke.piRNAcluster.20A.bed6.gz
Brennecke.piRNAcluster.42AB.bed6.gz
Brennecke.piRNAcluster.bed6.gz
Brennecke.piRNAcluster.flam.bed6.gz
BWAIndex
chrU.fa
cisNATs.bed.gz
cytoBand.txt
dm3.ChromInfo.txt
dm3.fa
dm3.fa.fai
dm3.gap.bed
dm3.genes.bed12
dm3.genes.fa
dm3.genes.gp
dm3.genes.gtf
dm3.genes+repBase+piRNACluster.gtf
dm3.genes+transposon+piRNACluster.gtf
dm3.gene+transposon.fa
dm3.hairpin.fa
dm3.mature.fa
dm3.mature.multiMapper.fa
dm3.piRNAcluster.bed.gz
dm3.piRNAcluster.fa
dm3.repBase.eref
dm3.repBase.fa
dm3.transposon.fa
dm3.transposon.rm.bed
Drosophila_melanogaster
Drosophila_melanogaster_UCSC_dm3.tar.gz
genomic_features
hpRNA.bed.gz
lincRNA.Young.bed6.gz
Log.out
mature2hairpin.allMapper.bed
mature2hairpin.multi.bed
mature2hairpin.uniq.bed
mrFastIndex
README.txt
repBase
rRNA.fa
STARIndex
structured_loci.bed.gz
UCSC.flyBase.3UTR.bed.gz
UCSC.flyBase.5UTR.bed.gz
UCSC.flyBase.CDS.bed.gz
UCSC.flyBase.Exons.bed.gz
UCSC.flyBase.Genes.bed12.gz
UCSC.flyBase.Genes.bed6.gz
UCSC.flyBase.Introns.bed.gz
UCSC.flyBase.Introns_xRM.bed.gz
UCSC.flyBase.Noncoding.bed.gz
UCSC.refSeq.3UTR.bed.gz
UCSC.refSeq.5UTR.bed.gz
UCSC.refSeq.CDS.bed.gz
UCSC.refSeq.Exon.bed.gz
UCSC.refSeq.Genes.bed12.gz
UCSC.refSeq.Intron.bed.gz
UCSC.RepeatMask.bed
UCSC.RepeatMask.inCluster.bed.gz
UCSC.RepeatMask.outCluster.bed.gz
UCSC.RepeatMask.Satellite.bed
UCSC.rRNA+tRNA+nonCoding.bed6.gz
UCSC.SimpleRepeats.bed.gz
unannotated_genome.bed.gz
variables
X-TAS.fa
Zamore.transposon.bed.gz
Zamore.transposon.group0.bed.gz
Zamore.transposon.group1.bed.gz
Zamore.transposon.group2.bed.gz
Zamore.transposon.group3.bed.gz
Zamore.transposon.inCluster.bed.gz
Zamore.transposon.outCluster.bed.gz

Is there someone else met the same trouble?
Please give me some suggestion.

Thanks very much!

Latest version of the genomes (danRer10)

Dear Dr.Han,

Thank you for providing a really nice pipeline for analyzing piRNA. Would it be possible to include the latest version of the zebrafish genome (danRer10, GRCz10)? I tried running piPipes with the version I installed and it is throwing error messages.

Thank you,
Neel

Error in RCircos.Validate.Genomic.Data(genomic.data, plot.type)

Dear Bo Han,

My run of piPipes dna ended as follows:

[2015-09-18 21:09:09 CEST] Discovering Variation by BreakDancer using BWA output
[2015-09-18 23:43:31 CEST] Draw Cisco plot to summarize the finding
Loading required package: RCircos

RCircos.Core.Components initialized.
Type ?RCircos.Reset.Plot.Parameters to see how to modify the core components.

Warning message:
In order(as.numeric(sub("chr", "", chr.names))) :
  NAs introduced by coercion
Some chromosomes are in genomic data only and have been removed.


Maximum lables for  chr4 is 2 . Extra ones are ignored.
Maximum lables for  chr2LHet is 0 . Extra ones are ignored.
Maximum lables for  chr2RHet is 5 . Extra ones are ignored.
Maximum lables for  chr3LHet is 4 . Extra ones are ignored.
Maximum lables for  chr3RHet is 4 . Extra ones are ignored.
Maximum lables for  chrXHet is 0 . Extra ones are ignored.

Some chromosomes are in genomic data only and have been removed.


Maximum lables for  chr4 is 2 . Extra ones are ignored.
Maximum lables for  chr2LHet is 0 . Extra ones are ignored.
Maximum lables for  chr2RHet is 5 . Extra ones are ignored.
Maximum lables for  chr3LHet is 4 . Extra ones are ignored.
Maximum lables for  chr3RHet is 4 . Extra ones are ignored.
Maximum lables for  chrXHet is 0 . Extra ones are ignored.

    Chr Start   End TransposonName TransposonDirection     Class VariantSupport
1 chr2L  7106  7606          TAHRE               sense        2p             43
2 chr2L 19524 20024          R1_DM               sense singleton              1
3 chr2L 23386 23886             FW           antisense singleton              1
4 chr2L 25651 25683           I_DM               sense       1p1            236
5 chr2L 26220 26720        BLOOD_I               sense singleton              1
6 chr2L 26500 27000     INVADER4_I           antisense singleton              1
  Frequency Junction1 Junction1Support Junction2 Junction2Support X5._Support
1    0.8269      7524               26      7524                0           0
2    0.0039     19774                0     19774                0           1
3    0.0042     23636                0     23636                0           1
4    0.8459     25661               13     25661                0         139
5    0.0030     26470                0     26470                0           1
6    0.0030     26750                0     26750                0           1
  X3._Support
1          17
2           0
3           0
4          84
5           0
6           0
Some chromosomes are in genomic data only and have been removed.

chr3RHet 2517343 2517843 
Error in RCircos.Validate.Genomic.Data(genomic.data, plot.type) : 
  Error! Location is outside of chromosome length.
Calls: RCircos.Tile.Plot ... RCircos.Get.Plot.Data -> RCircos.Validate.Genomic.Data
Execution halted
[2015-09-18 23:43:36 CEST] Finished running piPipes Genome-Seq pipeline version 1.0.1
[2015-09-18 23:43:36 CEST] ---------------------------------------------------------------------------------

The problem seems to correspond to the following line of the *.bwa-aln.insertion.refined.bp.summary file:

chr3RHet    2517343 2517843 Transib4    antisense   2p  7   1.0000  2517593 0   2517593 0   7   0

I'm using option -g dm3 for the genome.

biological replicates for smallRNA analysis

How to perform smallRNA analysis for biological replicates?

gene_transposon_cluster_direct_mapping is absent from piPipes rna output

The documentation for the RNA-Seq pipeline (https://github.com/bowhan/piPipes/wiki/RNA-seq) mentions a gene_transposon_cluster_direct_mapping directory that is not present in the output of piPipes rna.

sRNA single-sample mode alignment output

Hi! I've run the single-sample mode for 6 small RNA-seq libraries (triplicates in two conditions), and I want to do some downstream analyses outside of piPipes based on the quantification of small RNAs against various features (e.g. transposons, piRNA clusters). The workflow diagram at the bottom of the small RNA-seq page shows that the transcriptome mapping step done by bowtie produces SAM/BAM formatted output files, and that an abundance estimation step is performed with eXpress against transcripts, transposons, and piRNA clusters. I am interested in using both the bowtie SAM/BAM output files and the eXpress output files for further analysis, but I can't seem to find them in the directory produced by running piPipes. Can you tell me where these files might be located? Or whether I need to include a particular option in the piPipes call to obtain these output files? Thank you!

Error:mapping to rRNA failed / Could not locate a Bowtie index corresponding to basename "1"

Good afternoon,

I am having problems running the small RNA pipeline. I am trying to run the small RNA pipeline with the following command

piPipes small -i test.fastq.gz -g dm3 -o test2.out

But I got the following error

[2020-02-27 17:50:11 CST] sort is using:
/home/ch1005/piPipes/bin/sort
[2020-02-27 17:50:11 CST] md5sum is using:    
/usr/bin/md5sum
[2020-02-27 17:50:11 CST] awk is using:       
/home/ch1005/piPipes/bin/awk
[2020-02-27 17:50:11 CST] grep is using:      
/usr/bin/grep
[2020-02-27 17:50:11 CST] python is using:    
/opt/anaconda3/2019.10/bin/python
[2020-02-27 17:50:11 CST] samtools is using:  
/home/ch1005/piPipes/bin/samtools
[2020-02-27 17:50:11 CST] gs is using:        
/usr/bin/gs
[2020-02-27 17:50:11 CST] Rscript is using:   
/home/ch1005/R-3.6.1/bin/Rscript
[2020-02-27 17:50:11 CST] bowtie is using:    
/home/ch1005/piPipes/bin/bowtie
[2020-02-27 17:50:11 CST] ParaFly is using:   
/home/ch1005/piPipes/bin/ParaFly
[2020-02-27 17:50:11 CST] bedtools_piPipes is using:  
/home/ch1005/piPipes/bin/bedtools_piPipes
[2020-02-27 17:50:11 CST] bedGraphToBigWig is using:  
/home/ch1005/piPipes/bin/bedGraphToBigWig
[2020-02-27 17:50:11 CST] piPipes_bed2Summary is using:       
/home/ch1005/piPipes/bin/piPipes_bed2Summary
[2020-02-27 17:50:11 CST] piPipes_fastq_to_insert is using:  
/home/ch1005/piPipes/bin/piPipes_fastq_to_insert
[2020-02-27 17:50:11 CST] piPipes_insertBed_to_bed2 is using:         
/home/ch1005/piPipes/bin/piPipes_insertBed_to_bed2
[2020-02-27 17:50:11 CST] --------------------------------------------------------------------------------
[2020-02-27 17:50:11 CST] Beginning running [piPipes] small RNA pipeline single library mode version 1.1.0
[2020-02-27 17:50:11 CST] Converting fastq format into insert format
[2020-02-27 17:50:14 CST] Mapping to rRNA, with  mismatch(es) allowed
[2020-02-27 17:50:14 CST] Error: mapping to rRNA failed

Checking the rRNA.log I got the following

Could not locate a Bowtie index corresponding to basename "1"
Command: bowtie --wrapper basic-0 -r -S -v -k -p 8 --un input_read_files/test.x_rRNA.insert 1 rRNA input_read_files/test.insert

I would really appreciate if you can help me to solve this issue

terminate called after throwing an instance of 'std::length_error'

Hi,

I keep getting this error during the mapping phase of the small RNA pipeline.

terminate called after throwing an instance of 'std::length_error'
what(): basic_string::_S_create
(core dumped) piPipes_insertBed_to_bed2 $INPUT ${INSERT%.insert}.${GENOME}v${genome_MM}a.insert.bed > ${GENOME_ALLMAP_BED2}

could you help me out? thanks!

`bwtool` oom-killed but no obviously report by `piPipes chip`

Dear Bo Han,

I tried to execute the piPies chip examples from the wiki, and it seems that my computer ran out of memory and bwtool was killed:

$ dmesg -T | grep ill | tail -12
[Thu Jul  9 10:09:30 2015] bwtool invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0
[Thu Jul  9 10:09:30 2015]  [<ffffffff811526de>] oom_kill_process+0x1ce/0x330
[Thu Jul  9 10:09:30 2015] Out of memory: Kill process 21793 (bwtool) score 193 or sacrifice child
[Thu Jul  9 10:09:30 2015] Killed process 21793 (bwtool) total-vm:5524104kB, anon-rss:2192976kB, file-rss:28kB
[Thu Jul  9 10:14:49 2015] java invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
[Thu Jul  9 10:14:49 2015]  [<ffffffff811526de>] oom_kill_process+0x1ce/0x330
[Thu Jul  9 10:14:49 2015] Out of memory: Kill process 21861 (bwtool) score 184 or sacrifice child
[Thu Jul  9 10:14:49 2015] Killed process 21861 (bwtool) total-vm:5249072kB, anon-rss:1145860kB, file-rss:84kB
[Thu Jul  9 10:20:22 2015] bwtool invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
[Thu Jul  9 10:20:22 2015]  [<ffffffff811526de>] oom_kill_process+0x1ce/0x330
[Thu Jul  9 10:20:22 2015] Out of memory: Kill process 22557 (bwtool) score 216 or sacrifice child
[Thu Jul  9 10:20:22 2015] Killed process 22557 (bwtool) total-vm:6179804kB, anon-rss:4884292kB, file-rss:0kB

A kill occurred around 10:20, but I can't see obvious symptoms of that in the logs.

Looking at the full stdout log:

$ cat Hannon.ChIP.H3k9me3.nos_white.piPipes_m.stdout 
[2015-07-09 09:09:37 CEST] md5sum is using:     
/usr/bin/md5sum
[2015-07-09 09:09:37 CEST] awk is using:    
/usr/bin/awk
[2015-07-09 09:09:37 CEST] perl is using:   
/usr/bin/perl
[2015-07-09 09:09:37 CEST] python is using:     
/usr/bin/python
[2015-07-09 09:09:37 CEST] samtools is using:   
/home/bli/src/piPipes/bin/samtools
[2015-07-09 09:09:37 CEST] gs is using:     
/usr/bin/gs
[2015-07-09 09:09:37 CEST] Rscript is using:    
/usr/bin/Rscript
[2015-07-09 09:09:37 CEST] bowtie is using:     
/home/bli/src/piPipes/bin/bowtie
[2015-07-09 09:09:37 CEST] bowtie2 is using:    
/home/bli/src/piPipes/bin/bowtie2
[2015-07-09 09:09:37 CEST] csem is using:   
/home/bli/src/piPipes/bin/csem
[2015-07-09 09:09:37 CEST] ParaFly is using:    
/home/bli/src/piPipes/bin/ParaFly
[2015-07-09 09:09:37 CEST] bedtools_piPipes is using:   
/home/bli/src/piPipes/bin/bedtools_piPipes
[2015-07-09 09:09:37 CEST] bedGraphToBigWig is using:   
/home/bli/src/piPipes/bin/bedGraphToBigWig
[2015-07-09 09:09:37 CEST] express is using:    
/home/bli/src/piPipes/bin/express
[2015-07-09 09:09:37 CEST] macs2 is using:  
/home/bli/.local/bin/macs2
[2015-07-09 09:09:37 CEST] ---------------------------------------------------------------------------------
[2015-07-09 09:09:37 CEST] Beginning running [piPipes] ChIP-Seq pipeline version 1.2.1
[2015-07-09 09:09:37 CEST] Determining the version of fastQ using SolexaQA
[2015-07-09 09:09:37 CEST] Mapping IP reads to genome dm3 with Bowtie2
[2015-07-09 09:26:23 CEST] Mapping Input reads to genome dm3 with Bowtie2
[2015-07-09 09:31:14 CEST] Calling peaks with MACS2 and make enrichment bedGraph and bigWig
[2015-07-09 09:33:24 CEST] Aggregating signal on each genomic features
   No pages will be processed (FirstPage > LastPage).
   No pages will be processed (FirstPage > LastPage).
   No pages will be processed (FirstPage > LastPage).
[2015-07-09 10:12:44 CEST] Mapping to genes and transposon directly with Bowtie2
[2015-07-09 10:12:44 CEST] Direct mapping IP to transposon
[2015-07-09 10:14:53 CEST] Direct mapping INPUT to transposon
[2015-07-09 10:16:35 CEST] Making summary graph


All commands completed successfully. :-)

[2015-07-09 10:16:53 CEST] quantification IP using eXpress
[2015-07-09 10:22:31 CEST] quantification input using eXpress
[2015-07-09 10:26:14 CEST] Finished running piPipes ChIP-Seq pipeline version 1.2.1
[2015-07-09 10:26:14 CEST] ---------------------------------------------------------------------------------

Searching for around 10:20 in the stderr log:

$ cat Hannon.ChIP.H3k9me3.nos_white.piPipes_m.stderr | grep "10:2[0,1,2,3]"
2015-Jul-09 10:20:10 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:20:10 - 
2015-Jul-09 10:20:10 - Processing input fragment alignments...
2015-Jul-09 10:20:24 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:20:24 - 
2015-Jul-09 10:20:24 - Processing input fragment alignments...
2015-Jul-09 10:20:38 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:20:38 - 
2015-Jul-09 10:20:38 - Processing input fragment alignments...
2015-Jul-09 10:20:52 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:20:52 - 
2015-Jul-09 10:20:52 - Processing input fragment alignments...
2015-Jul-09 10:21:06 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:21:06 - 
2015-Jul-09 10:21:06 - Processing input fragment alignments...
2015-Jul-09 10:21:21 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:21:21 - 
2015-Jul-09 10:21:21 - Processing input fragment alignments...
2015-Jul-09 10:21:35 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:21:35 - 
2015-Jul-09 10:21:35 - Processing input fragment alignments...
2015-Jul-09 10:21:49 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:21:49 - 
2015-Jul-09 10:21:49 - Processing input fragment alignments...
2015-Jul-09 10:22:03 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:22:03 - 
2015-Jul-09 10:22:03 - Processing input fragment alignments...
2015-Jul-09 10:22:17 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:22:17 - 
2015-Jul-09 10:22:17 - Processing input fragment alignments...
2015-Jul-09 10:22:31 - COMPLETED: Processed 653506 mapped fragments, targets are in 104 bundles.
2015-Jul-09 10:22:31 - Writing results to file...
2015-Jul-09 10:22:31 - Done.
2015-Jul-09 10:22:31 - Attempting to read '/home/bli/Documents/Equipe_Chambeyron/ChIP-seq/Hannon.ChIP.H3K9me3.nos_white.piPipes_m_out/direct_mapping/Hannon.ChIP.H3K9me3.nos_white_1.input.transposon.bam' in BAM format...
2015-Jul-09 10:22:31 - Parsing BAM header...
2015-Jul-09 10:22:31 - Loading target sequences and measuring bias background...
2015-Jul-09 10:22:31 - Initialized 126 targets.
2015-Jul-09 10:22:31 - Processing input fragment alignments...
2015-Jul-09 10:22:38 - Synchronized auxiliary parameter tables.
2015-Jul-09 10:22:39 - Synchronized auxiliary parameter tables.
2015-Jul-09 10:22:40 - Synchronized auxiliary parameter tables.
2015-Jul-09 10:22:40 - Synchronized auxiliary parameter tables.
2015-Jul-09 10:22:41 - Synchronized auxiliary parameter tables.
2015-Jul-09 10:22:42 - Synchronized auxiliary parameter tables.
2015-Jul-09 10:22:42 - Waiting for auxiliary parameter update to complete...
2015-Jul-09 10:22:42 - COMPLETED: Processed 139177 mapped fragments, targets are in 106 bundles.
2015-Jul-09 10:22:42 - WARNING: Not enough fragments observed to accurately learn bias parameters. Either disable bias correction (--no-bias-correct) or provide a file containing auxiliary parameters (--aux-param-file).
2015-Jul-09 10:22:42 - 
2015-Jul-09 10:22:42 - Processing input fragment alignments...
2015-Jul-09 10:22:52 - COMPLETED: Processed 139177 mapped fragments, targets are in 106 bundles.
2015-Jul-09 10:22:52 - 
2015-Jul-09 10:22:52 - Processing input fragment alignments...
2015-Jul-09 10:23:02 - COMPLETED: Processed 139177 mapped fragments, targets are in 106 bundles.
2015-Jul-09 10:23:02 - 
2015-Jul-09 10:23:02 - Processing input fragment alignments...
2015-Jul-09 10:23:12 - COMPLETED: Processed 139177 mapped fragments, targets are in 106 bundles.
2015-Jul-09 10:23:12 - 
2015-Jul-09 10:23:12 - Processing input fragment alignments...
2015-Jul-09 10:23:22 - COMPLETED: Processed 139177 mapped fragments, targets are in 106 bundles.
2015-Jul-09 10:23:22 - 
2015-Jul-09 10:23:22 - Processing input fragment alignments...
2015-Jul-09 10:23:33 - COMPLETED: Processed 139177 mapped fragments, targets are in 106 bundles.
2015-Jul-09 10:23:33 - 
2015-Jul-09 10:23:33 - Processing input fragment alignments...
2015-Jul-09 10:23:43 - COMPLETED: Processed 139177 mapped fragments, targets are in 106 bundles.
2015-Jul-09 10:23:43 - 
2015-Jul-09 10:23:43 - Processing input fragment alignments...
2015-Jul-09 10:23:53 - COMPLETED: Processed 139177 mapped fragments, targets are in 106 bundles.
2015-Jul-09 10:23:53 - 
2015-Jul-09 10:23:53 - Processing input fragment alignments...

I'm not sure I looked at the correct section of the stderr log, though: the man page for dmesg says that option -T may produce inaccurate timestamps.

Segmentation Fault

I'm seeing these errors when I run piPipes:
piPipes_smallRNA_bed2_to_bw.sh: line 32: 13962 Segmentation fault piPipes_bed2_to_bedGraph -i $OUTDIR/${INPUT_BED_NAME%bed2}sorted.bed2 -c $CHROM -p $CPU -o $OUTDIR/${INPUT_BED_NAME%bed2}sorted.

piPipes_smallRNA_bed2_to_bw.sh: line 33: 13965 Segmentation fault piPipes_bed2_to_bedGraph -i $OUTDIR/${INPUT_BED_NAME%bed2}sorted.uniq.bed2 -c $CHROM -p $CPU -o $OUTDIR/${INPUT_BED_NAME%bed2}sorted.uniq.

As far as I can tell, these are the only errors thrown. The transposon_piRNAcluster_mapping_normalized_by_mirna and the bigWig_normalized_by_mirna folders are empty. Any ideas?

Errors when launching piPipes small RNA

hello,
I have some issues when I launch piPipes small RNA analysis.

in piPipes.stderr file, I have this lane, telling that the "variables" file is not found :
"piPipes_smallRNA.sh: line 158: piPipes-master/common/hg19/variables: Aucun fichier ou dossier de ce type". I wanted to know where to find this file.

in piPipes.stdout file, I have :
"[31;1m[2015-01-12 14:32:07 CET] Error: length for siRNA is not defined! please check the "variable" file under common$GENOME". Again, what is this "variable" file.

Thank you

install custom genome error

Dear Bo Han,

I am trying to install orycun2 genome which is without iGenome support, I have provided all the files that you have mentioned: genome sequence in fasta format, transposon sequence in fasta format, piRNA cluster in bed format, genes annotation in gtf format, miRNA hairpin sequence in fasta format, miRNA sequence in fasta format. However, there was a error when performing repBase transposon annotation:
[2023-02-16 16:30:58 CST] Begining installing the genome orycun2
[2023-02-16 16:30:58 CST] Testing/Installing missing R packages
[2023-02-16 16:30:58 CST] Preparing genomic sequence/annotation and making indexes
[2023-02-16 16:30:58 CST] Building STAR index for genome
[2023-02-16 16:30:58 CST] Building mrFast index for genome
[2023-02-16 16:30:58 CST] Building Bowtie/Bowtie2 index for rRNA
[2023-02-16 16:30:58 CST] Building index for microRNA hairpin, still using fasta from mirBase
[2023-02-16 16:30:58 CST] Building Bowtie/BWA index for repBase transposon annotation
Can't open repBase/L2a::LINE/L2::chr1:51-174.fa to write: No such file or directory
[2023-02-16 16:30:58 CST] Building Bowtie/BWA index for piRNA cluster
[2023-02-16 16:30:58 CST] Building Bowtie/BWA index for repBase + piRNA cluster + genes
Do you have any idea about that?

By the way, could piPipes handle multiple sample for small RNA-Seq and RNA-Seq, thank you!

Yuan

Error: mapping to rRNA failed

Dear Dr.Han,

I have a problem about mapping when I used the small RNA pipeline. I follow the "install" and install the genome. But I failed in mapping and I don't know how to solve it.
Here is my code and program.

$ piPipes small -i Sample_A.fastq -g mm9 -c 24 -o output
[2016-12-30 23:56:48 CST] sort is using:
/media/heming/新加卷/seq/pirna/piPipes/bin/sort
[2016-12-30 23:56:48 CST] md5sum is using:
/usr/bin/md5sum
[2016-12-30 23:56:48 CST] awk is using:
/usr/bin/awk
[2016-12-30 23:56:48 CST] grep is using:
/bin/grep
[2016-12-30 23:56:48 CST] python is using:
/usr/bin/python
[2016-12-30 23:56:48 CST] samtools is using:
/media/heming/新加卷/seq/pirna/piPipes/bin/samtools
[2016-12-30 23:56:48 CST] gs is using:
/usr/bin/gs
[2016-12-30 23:56:48 CST] Rscript is using:
/usr/bin/Rscript
[2016-12-30 23:56:48 CST] bowtie is using:
/media/heming/新加卷/seq/pirna/piPipes/bin/bowtie
[2016-12-30 23:56:48 CST] ParaFly is using:
/media/heming/新加卷/seq/pirna/piPipes/bin/ParaFly
[2016-12-30 23:56:48 CST] bedtools_piPipes is using:
/media/heming/新加卷/seq/pirna/piPipes/bin/bedtools_piPipes
[2016-12-30 23:56:48 CST] bedGraphToBigWig is using:
/media/heming/新加卷/seq/pirna/piPipes/bin/bedGraphToBigWig
[2016-12-30 23:56:48 CST] piPipes_bed2Summary is using:
/media/heming/新加卷/seq/pirna/piPipes/bin/piPipes_bed2Summary
[2016-12-30 23:56:48 CST] piPipes_fastq_to_insert is using:
/media/heming/新加卷/seq/pirna/piPipes/bin/piPipes_fastq_to_insert
[2016-12-30 23:56:48 CST] piPipes_insertBed_to_bed2 is using:
/media/heming/新加卷/seq/pirna/piPipes/bin/piPipes_insertBed_to_bed2
[2016-12-30 23:56:48 CST] ---------------------------------------------------------------------------------
[2016-12-30 23:56:48 CST] Beginning running [piPipes] small RNA pipeline single library mode version 1.1.0
[2016-12-30 23:56:48 CST] Converting fastq format into insert format
[2016-12-30 23:56:48 CST] Mapping to rRNA, with mismatch(es) allowed
[2016-12-30 23:56:49 CST] Error: mapping to rRNA failed

Thank you,
heming

bigWig in binary code

Hey i was using the piPipes small, and in the output folder 'bigWig_normalized_by_mirna' all the bigwig files are in binary code.
Can u make the output in standard utf-8 or something?
Thank you

Errors while installing R dependencies

Dear Bo Han,

I get the following errors when running piPipes install -g dm3 -c $(nproc):

ERROR: dependencies ‘plyr’, ‘reshape2’, ‘scales’ are not available for package ‘ggplot2’
* removing ‘/home/bli/src/piPipes/Rlib/ggplot2’
ERROR: dependencies ‘ggplot2’, ‘scales’ are not available for package ‘Hmisc’
* removing ‘/home/bli/src/piPipes/Rlib/Hmisc’

The downloaded source packages are in
    ‘/tmp/Rtmp7KPCan/downloaded_packages’
Loading required package: ggplot2
Error in pkgTest("ggplot2") : 
  Failed to install the package. Please check the internet access or update your R if it is too old.
In addition: Warning messages:
1: In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE,  :
  there is no package called ‘ggplot2’
2: In install.packages(x, dep = TRUE, lib = paste(Sys.getenv("PIPELINE_DIRECTORY"),  :
  installation of package ‘reshape2’ had non-zero exit status
3: In install.packages(x, dep = TRUE, lib = paste(Sys.getenv("PIPELINE_DIRECTORY"),  :
  installation of package ‘scales’ had non-zero exit status
4: In install.packages(x, dep = TRUE, lib = paste(Sys.getenv("PIPELINE_DIRECTORY"),  :
  installation of package ‘ggplot2’ had non-zero exit status
5: In install.packages(x, dep = TRUE, lib = paste(Sys.getenv("PIPELINE_DIRECTORY"),  :
  installation of package ‘Hmisc’ had non-zero exit status
6: In library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE,  :
  there is no package called ‘ggplot2’
Execution halted
[2015-07-03 16:51:41 CEST] Downloading iGenome dm3

(It then continues downloading dm3 genome)

My version of R is 3.0.2 (It's the one packaged in Ubuntu 14.04 http://packages.ubuntu.com/fr/trusty/r-base-core). Could it be too old ?

Other remarks:

Later I get a bunch of warnings:
WARNING. chromosome (chrUextra) was not found in the FASTA file. Skipping.
Just before
[2015-07-03 17:09:14 CEST] Building STAR index for genome
I noticed a typo in the configuration questions: "trasnposons" on line 159 of bin/piPipes_install_genomes.sh

Error: unable to find or generate fasta for the genome

I got an error here when I first install the genome : Error: Failed to download the genome file, please check the internet.
then I re-installed the genome and it appears :
[2017-11-10 10:27:18 CST] Downloading iGenome mm10
[2017-11-10 10:27:18 CST] Uncompressing genome mm10
[2017-11-10 10:27:18 CST] Preparing genomic sequence/annotation and making indexes
[2017-11-10 10:27:18 CST] Error: unable to find or generate fasta for the genome.
So, how should I do to re-install it.
thanks

problem with piPipes installation on OSx

Hi,

I installed everything require for the piPipes running following the wiki you made. I even test the installation on a Macbook pro with another OS version, but I still got the same message on my terminal when I want to test if piPipes is correctly installed.
Can you help me with this issue?

Thanks,

Marine

Include a modules file for cluster users

I am running piPipes on the Massachusetts GHPCC. Certain components of the installation won't proceed without first loading the right modules. Would it be possible to create a file modules.txt simply consisting of the modules and versions used to develop this software?

parameters in `/common/mm9/variables`

Hi, Could you please give me an example of the parameters in /common/hg19/variables ? I'm a new starter and not very sure about the numbers I should use. Thank you so much !

Problems with R packages(ggplot2?)

Dear Dr.Han,

I'm having troubles with what it seems to be related with R packages. When trying to run the pipeline I'm getting this warnings and some of the graphics on the output are strange.

Warning messages:
1: `axis.ticks.margin` is deprecated. Please set `margin` property  of `axis.text` instead 
2: `panel.margin` is deprecated. Please use `panel.spacing` property instead 
3: `legend.margin` must be specified using `margin()`. For the old behavior use legend.spacing 
Error in read.table(file, FALSE) : no lines available in input
Calls: draw_smRNA_lendis -> read.table
Execution halted

I searched on the internet and it may be related with some updates in the ggplot2 package, but I'm not sure. I appreciate any help and thank you in advance for your attention.

Best Regards,
Tarcísio Brito

Strange name in piPipes chip2 example output

Dear Bo Han,

I ran the chip2 example from the wiki, and I notice that some of the output files have names prefixed by "nos_." and some have double dots before file extension:

$ ls -1 Hannon.ChIP.H3K9me3.nos_white.vs.nos_piwi/differential_peaks_calling
nos_.nos_white_vs_nos_piwi.bdgdiff.log
nos_.nos_white_vs_nos_piwi_c3.0_common.bed
nos_.nos_white_vs_nos_piwi_c3.0_cond1.bed
nos_.nos_white_vs_nos_piwi_c3.0_cond2.bed
$ ls -1 Hannon.ChIP.H3K9me3.nos_white.vs.nos_piwi/aggregate_output
nos_.nos_white_vs_nos_piwi_c3.0_cond1.ends.txt
nos_.nos_white_vs_nos_piwi_c3.0_cond1.meta.txt
nos_.nos_white_vs_nos_piwi_c3.0_cond1.starts.txt
nos_.nos_white_vs_nos_piwi_c3.0_cond2.ends.txt
nos_.nos_white_vs_nos_piwi_c3.0_cond2.meta.txt
nos_.nos_white_vs_nos_piwi_c3.0_cond2.starts.txt
$ ls -1 Hannon.ChIP.H3K9me3.nos_white.vs.nos_piwi/pdfs
nos_.nos_white_vs_nos_piwi_c3.0_cond1.ends.pdf
nos_.nos_white_vs_nos_piwi_c3.0_cond1.meta.pdf
nos_.nos_white_vs_nos_piwi_c3.0_cond1.starts..pdf
nos_.nos_white_vs_nos_piwi_c3.0_cond2.ends.pdf
nos_.nos_white_vs_nos_piwi_c3.0_cond2.meta.pdf
nos_.nos_white_vs_nos_piwi_c3.0_cond2.starts..pdf
nos_white_nos_piwi.direct_mapping.unique.pdf

I ran the command as follows:

$ piPipes chip2 \
> -a Hannon.ChIP.H3K9me3.nos_white.piPipes_out \
> -b Hannon.ChIP.H3K9me3.nos_piwi.piPipes_out \
> -g dm3 \
> -c 3 \
> -o Hannon.ChIP.H3K9me3.nos_white.vs.nos_piwi \
> -A nos_white \
> -B nos_piwi \
> 1> Hannon.ChIP.H3K9me3.nos_white.vs.nos_piwi.stdout \
> 2> Hannon.ChIP.H3K9me3.nos_white.vs.nos_piwi.stderr

Missing columns in *.transposon.sorted.unique.bigWig.summary

Dear Bo Han,

In one of my analyses, the *.transposon.unique.pdf file is never generated by piPipes chip

I could trace the problem in the error log:

no data in region FBgn0046701_Penelope:0-804 in /home/severine.chambeyron/projet_Abdou/ChIP-Seq/piPipes_chip/shpiwi_rep1/direct_mapping/Chambeyron_8_4_IP_H3K9me3_shpiwi_rep1_S28_L006_R1_001.input.transposon.sorted.unique.bigWig

no data in region FBgn0067418_Helitron:0-564 in /home/severine.chambeyron/projet_Abdou/ChIP-Seq/piPipes_chip/shpiwi_rep1/direct_mapping/Chambeyron_8_4_IP_H3K9me3_shpiwi_rep1_S28_L006_R1_001.input.transposon.sorted.unique.bigWig

Loading required package: parallel
Error in scan(file, what, nmax, sep, dec, quote, skip, nlines, na.strings,  : 
  line 580255 did not have 4 elements
Calls: read.table -> scan
Execution halted
2016-May-04 17:47:17 - Attempting to read '/home/severine.chambeyron/projet_Abdou/ChIP-Seq/piPipes_chip/shpiwi_rep1/direct_mapping/Chambeyron_8_4_IP_H3K9me3_shpiwi_rep1_S28_L006_R1_001.IP.transposon.bam' in BAM format...

The problematic zone in the *.transposon.sorted.unique.bigWig file is the following:

FBgn0067418_Helitron    551 0.0000000   0.0000000
FBgn0067418_Helitron    552 0.0000000   0.0000000
FBgn0067418_Helitron    553 0.0000000   0.0000000
FBgn0067418_Helitron    554 0.0000000   0.0000000
FBgn0067418_Helitron    555 0.0000000   0.0000000
FBgn0067418_Helitron    556 0.0000000   0.0000000
FBgn0067418_Helitron    557 0.0000000   0.0000000
FBgn0067418_Helitron    558 0.0000000   0.0000000
FBgn0067418_Helitron    559 0.0000000   0.0000000
FBgn0067418_Helitron    560 0.0000000   0.0000000
FBgn0067418_Helitron    561 0.0000000   0.0000000
FBgn0067418_Helitron    562 0.0000000   0.0000000
FBgn0067418_Helitron    563 0.0000000   0.0000000
FBgn0067418_Helitron    564 0.0000000   0.0000000
    FBgn0067418_Helitron    565
    FBgn0067418_Helitron    566
    FBgn0067418_Helitron    567
    FBgn0067418_Helitron    568
    FBgn0067418_Helitron    569
    FBgn0067418_Helitron    570
    FBgn0067418_Helitron    571
    FBgn0067418_Helitron    572
    FBgn0067418_Helitron    573
    FBgn0067418_Helitron    574
    FBgn0067418_Helitron    575

The problem does not happen with data from another replicate of the same experiment.
Here is the corresponding error log portion:

Number of  Commands: 252
no data in region FBgn0046701_Penelope:0-804 in /home/severine.chambeyron/projet_Abdou/ChIP-Seq/piPipes_chip/shpiwi_rep2/direct_mapping/Chambeyron_12_8_IP_H3K9me3_shpiwi_rep2_S32_L006_R1_001.IP.transposon.sorted.unique.bigWig

no data in region FBgn0046701_Penelope:0-804 in /home/severine.chambeyron/projet_Abdou/ChIP-Seq/piPipes_chip/shpiwi_rep2/direct_mapping/Chambeyron_12_8_IP_H3K9me3_shpiwi_rep2_S32_L006_R1_001.input.transposon.sorted.unique.bigWig

Loading required package: parallel
$FBgn0000004_17.6
null device 
          1 

$FBgn0000005_297

In the corresponding *.transposon.sorted.unique.bigWig file, the "Helitron" zone stops at position 564 instead of continuing with missing data in the problematic replicate:

FBgn0067418_Helitron    551 0.1344920   0.0000000
FBgn0067418_Helitron    552 0.1344920   0.0000000
FBgn0067418_Helitron    553 0.1344920   0.0000000
FBgn0067418_Helitron    554 0.1008690   0.0000000
FBgn0067418_Helitron    555 0.1008690   0.0000000
FBgn0067418_Helitron    556 0.1008690   0.0000000
FBgn0067418_Helitron    557 0.1008690   0.0000000
FBgn0067418_Helitron    558 0.1008690   0.0000000
FBgn0067418_Helitron    559 0.1008690   0.0000000
FBgn0067418_Helitron    560 0.1008690   0.0000000
FBgn0067418_Helitron    561 0.0672460   0.0000000
FBgn0067418_Helitron    562 0.0672460   0.0000000
FBgn0067418_Helitron    563 0.0672460   0.0000000
FBgn0067418_Helitron    564 0.0336230   0.0000000
FBgn0067419_G7  1   0.4370990   0.0000000
FBgn0067419_G7  2   0.5043450   0.0000000
FBgn0067419_G7  3   0.5043450   0.0000000
FBgn0067419_G7  4   0.5379680   0.0000000
FBgn0067419_G7  5   0.6388370   0.0000000
FBgn0067419_G7  6   0.7060830   0.0000000
FBgn0067419_G7  7   0.7733290   0.0000000
FBgn0067419_G7  8   0.7733290   0.0000000
FBgn0067419_G7  9   0.8069520   0.0000000
FBgn0067419_G7  10  0.9078210   0.0000000
FBgn0067419_G7  11  1.0086900   0.0000000

Could the problem be related to the "no data in region" warnings ?

There were no mention of missing data for Helitron in the error log for the well-behaving replicate.
For the problematic replicate, there was a mention of missing data in the Helitron zone preceding the wrong lines., but only in the input, not in the IP.

piPipes_smallRNA_bed2_to_bw.sh: line 32: 21799 Segmentation fault

I'm trying to run a smallRNA pipeline before running a degradome-seq pipeline. I've installed the ce10 genome and am running this on the MGHPCC cluster, which uses an x86_64 architecture.

When I run the smallRNA pipeline, it finishes in 0.422 seconds, and the error output says that there are two segmentation faults when running piPipes_smallRNA_bed2_to_bw.sh. Specifically, this occurs when running the binary piPipes_bed2_to_bedGraph.

Below is the full output:

[ed40a@ghpcc06 degradome_seq]$ time ~/software/piPipes/piPipes small -i smallRNA_datasets/N2_all/N2_all.fa -g ce10 -o ./piPipes/smallRNA/N2_all -c 8
[2018-07-25 14:16:59 EDT] sort is using: 	
/home/ed40a/software/piPipes/bin/sort
[2018-07-25 14:16:59 EDT] md5sum is using: 	
/usr/bin/md5sum
[2018-07-25 14:16:59 EDT] awk is using: 	
/bin/awk
[2018-07-25 14:16:59 EDT] grep is using: 	
/bin/grep
[2018-07-25 14:16:59 EDT] python is using: 	
/share/pkg/python/2.7.9/bin/python
[2018-07-25 14:16:59 EDT] samtools is using: 	
/home/ed40a/software/piPipes/bin/samtools
[2018-07-25 14:16:59 EDT] gs is using: 	
/usr/bin/gs
[2018-07-25 14:16:59 EDT] Rscript is using: 	
/share/pkg/R/3.5.0/bin/Rscript
[2018-07-25 14:16:59 EDT] bowtie is using: 	
/home/ed40a/software/piPipes/bin/bowtie
[2018-07-25 14:16:59 EDT] ParaFly is using: 	
/home/ed40a/software/piPipes/bin/ParaFly
[2018-07-25 14:16:59 EDT] bedtools_piPipes is using: 	
/home/ed40a/software/piPipes/bin/bedtools_piPipes
[2018-07-25 14:16:59 EDT] bedGraphToBigWig is using: 	
/home/ed40a/software/piPipes/bin/bedGraphToBigWig
[2018-07-25 14:16:59 EDT] piPipes_bed2Summary is using: 	
/home/ed40a/software/piPipes/bin/piPipes_bed2Summary
[2018-07-25 14:16:59 EDT] piPipes_fastq_to_insert is using: 	
/home/ed40a/software/piPipes/bin/piPipes_fastq_to_insert
[2018-07-25 14:16:59 EDT] piPipes_insertBed_to_bed2 is using: 	
/home/ed40a/software/piPipes/bin/piPipes_insertBed_to_bed2
[2018-07-25 14:16:59 EDT] ---------------------------------------------------------------------------------
[2018-07-25 14:16:59 EDT] Beginning running [piPipes] small RNA pipeline single library mode version 1.1.0
[2018-07-25 14:16:59 EDT] Converting fastq format into insert format
[2018-07-25 14:16:59 EDT] Mapping to rRNA, with 2 mismatch(es) allowed
[2018-07-25 14:16:59 EDT] Mapping to microRNA Hairpin, with 1 mismatch(es) allowed; only keep unique mappers
[2018-07-25 14:16:59 EDT] Calculate microRNA heterogeneity
[2018-07-25 14:16:59 EDT] Mapping to genome, with 1 mismatch(es) allowed
[2018-07-25 14:16:59 EDT] Separating unique and multiple mappers
[2018-07-25 14:16:59 EDT] Separating siRNA, piRNA based on length
[2018-07-25 14:16:59 EDT] Plotting length distribution
[2018-07-25 14:16:59 EDT] Intersecting with genomic features, make length distribution, nucleotide fraction for siRNA/piRNA assigned to each feature
[2018-07-25 14:16:59 EDT] Making bigWig files for genome browser
/home/ed40a/software/piPipes/bin/piPipes_smallRNA_bed2_to_bw.sh: line 32: 43365 Segmentation fault      piPipes_bed2_to_bedGraph -i $OUTDIR/${INPUT_BED_NAME%bed2}sorted.bed2 -c $CHROM -p $CPU -o $OUTDIR/${INPUT_BED_NAME%bed2}sorted.
/home/ed40a/software/piPipes/bin/piPipes_smallRNA_bed2_to_bw.sh: line 33: 43368 Segmentation fault      piPipes_bed2_to_bedGraph -i $OUTDIR/${INPUT_BED_NAME%bed2}sorted.uniq.bed2 -c $CHROM -p $CPU -o $OUTDIR/${INPUT_BED_NAME%bed2}sorted.uniq.
[2018-07-25 14:16:59 EDT] Direct mapping to transposon and piRNA cluster and make distribution plot
[2018-07-25 14:16:59 EDT] Merging pdfs
[2018-07-25 14:16:59 EDT] Finished running piPipes small RNA pipeline version 1.1.0
[2018-07-25 14:16:59 EDT] ---------------------------------------------------------------------------------

real	0m0.422s
user	0m0.109s
sys	0m0.155s

I'm noticing other users are experiencing this same issue. Has anyone been able to fix this? What is the reason for the error, and what are potential workarounds?

Error in which(x <= 10^floor(log10(x)) * nice)[[1]] : subscript out of bounds

when i run /piPipes rnaseq2, the program was interrupted. The error message is:
Loading required package: gdata
gdata: read.xls support for 'XLS' (Excel 97-2004) files ENABLED.

gdata: Unable to load perl libaries needed by read.xls()
gdata: to support 'XLSX' (Excel 2007+) files.

gdata: Run the function 'installXLSXsupport()'
gdata: to automatically download and install the perl
gdata: libaries needed to support Excel XLS and XLSX formats.

Attaching package: ‘gdata’

The following object is masked from ‘package:stats’:

nobs

The following object is masked from ‘package:utils’:

object.size

The following object is masked from ‘package:base’:

startsWith

Loading required package: ggplot2
Loading required package: grid
Loading required package: ggthemes
Loading required package: scales
Error in which(x <= 10^floor(log10(x)) * nice)[[1]] :
subscript out of bounds
Calls: roundUp
In addition: Warning messages:
1: In max(sample$eff_counts.x, sample$eff_counts.y) :
no non-missing arguments to max; returning -Inf
2: In roundUp(10 * (max(sample$eff_counts.x, sample$eff_counts.y))) :
NaNs produced
3: In which(x <= 10^floor(log10(x)) * nice) : NaNs produced
Execution halted

Should I change any parameters? Thank you very much!

When installing genome, stuck in 'receiving incremental file list'

Executed:
pipipes install -g hg19 -c 8
The genome (hg19) was downloaded, but for about 48 hrs. I get (tail of the output file):
sideBwtSz: 48
sideBwtLen: 192
numSides: 812949
numLines: 812949
ebwtTotLen: 52028736
ebwtTotSz: 52028736
color: 0
reverse: 1
Total time for backward call to driver() for mirror index: 00:05:12
receiving incremental file list

piRNA plots python error

I am tryng to run pipipes small on trimmed small RNAseq data but
When running the pipeline in get the following error:
Calls: draw_smRNA_percentage -> read.table
Execution halted
File "/usr/people/bioc1647/piPipes/bin/piPipes_nuc_percentage.py", line 33
print str (A) + '\t' + str (C) + '\t' + str (G) + '\t' + str (T)
^
SyntaxError: invalid syntax
File "/usr/people/bioc1647/piPipes/bin/piPipes_nuc_percentage.py", line 33
print str (A) + '\t' + str (C) + '\t' + str (G) + '\t' + str (T)
^
SyntaxError: invalid syntax
File "/usr/people/bioc1647/piPipes/bin/piPipes_nuc_percentage.py", line 33
print str (A) + '\t' + str (C) + '\t' + str (G) + '\t' + str (T)
^
SyntaxError: invalid syntax
File "/usr/people/bioc1647/piPipes/bin/piPipes_nuc_percentage.py", line 33
print str (A) + '\t' + str (C) + '\t' + str (G) + '\t' + str (T)
^
SyntaxError: invalid syntax

Warning messages:
1: axis.ticks.margin is deprecated. Please set margin property of axis.text instead
2: panel.margin is deprecated. Please use panel.spacing property instead
3: legend.margin must be specified using margin(). For the old behavior use legend.spacing

Unable to download the human genome from igenome

2021-04-19 06:16:29 IST] Downloading iGenome in Download ONLY mode.
--2021-04-19 06:16:29-- ftp://igenome:*password*@ussd-ftp.illumina.com/Homo_sapiens/UCSC/hg19/Homo_sapiens_UCSC_hg19.tar.gz
=> ‘Homo_sapiens_UCSC_hg19.tar.gz’
Resolving ussd-ftp.illumina.com (ussd-ftp.illumina.com)... 192.84.34.14
Connecting to ussd-ftp.illumina.com (ussd-ftp.illumina.com)|192.84.34.14|:21... connected.
Logging in as igenome ...
Error in server response, closing control connection.
Retrying.

I am unable to download the human genome hg19, as the ftp is unable to login. Is there any alternative to it?

piPipes

Hi,
I went through the installation but I am stuck now. Whatever I type (pipipes, or piPipes_debug) this is the error I get:
readlink: illegal option -- f
usage: readlink [-n] [file ...]
usage: dirname path
bash: /piPipes.sh: No such file or directory

and this is the output of which piPipes
/Users/user/bin/piPipes/piPipes

I hope someone can help me,
Thanks

bowhan / pipipes Goto Github PK

pipipes's People

Contributors

Stargazers

Watchers

Forkers

pipipes's Issues

Recommend Projects

Recommend Topics

Recommend Org