Coder Social home page Coder Social logo

hichipper's Introduction




Build Status PyPI version Documentation Status License: MIT DOI Join the chat at https://gitter.im/aryeelab/hichipper

This package is maintained by Caleb Lareau in the Aryee Lab. Source code is made freely available here and a packaged install version is provided through PyPi.

About

The hichipper package implements our data processing and quality control pipeline for HiChIP data. This package takes output from a HiC-Pro run and a sample manifest file (.yaml) that coordinates optional high-quality peaks (identified through ChIP-Seq) and restriction fragment locations (see folder here) as input and produces output that can be used to 1) determine library quality, 2) identify and characterize DNA loops and 3) interactively visualize loops. Loops are assigned strength and confidence metrics that can be used to evaluate samples individually or for differential analysis in downstream tools like diffloop.

We have used the library QC metrics with as few as 1 million reads, enabling library quality to be assessed through shallow (and cheap) sequencing before performing a full depth sequencing run.

A graphical overview showing how hichipper integrates with other tools in the analysis of raw HiChIP data is shown in the overview figure below. Detailed descriptions of the different branches of input, output, and configuration for hichipper are discussed in the documentation.

big1 big2

Installation/Documentation/FAQ/.etc

Check out the hichipper documentation for detailed installation instructions, dependency configuration, and other information regarding the tool and its implementation.

Workflow Overview

A simple graphical guide to processing HiChIP data is shown below. The role of hichipper is to import aligned read files from (e.g. HiC-Pro) as well as location of restriction fragment files (available here) coordinated through a .yaml configuration file and produce user-friendly output.

In particular, hichipper allows users to pre-supply their own set of gold-standard peaks (e.g. from ChIP-Seq) or call peaks directly from HiChIP data using a novel background detection algorithm. In either case, interactions and chromatin loops can be called using a restriction fragment-aware approach that substantially increases read density in loops.

hichipper_overview

Quality control reports

In the qcReports folder, we collect the .html QC report files associated with text annotations from the experiments performed in the original HiChIP manuscript as well as other reports generated by anonymous collaborators that demonstrate libraries that did not prepare well, likely due to poor in situ ligation. To determine the quality of a new HiChIP library, we recommend comparing the vital statistics and interactive tables and figures between existing libraries.

Citation

If you use hichipper for your work, please cite:

Lareau, C.A. and Aryee, M.J. (2018) "hichipper: A preprocessing pipeline for assessing library quality and DNA loops from HiChIP data." bioRxiv doi: https://doi.org/10.1101/192302

Questions/comments/feedback

are always welcomed. The easiest way for us to have correspondence (if appropriate/interesting for the public) is through raising a new issue or our hichipper chat room on gitter. Otherwise, email Caleb with any private concerns.

hichipper logo made with autowide font with these blue and red color definitions.

hichipper's People

Contributors

0xaf1f avatar areyesq89 avatar caleblareau avatar gitter-badger avatar martinaryee avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

hichipper's Issues

split files in hicpro

Hi,
I use macOS. I run hichipper after hicpro. In hicpro I split my raw data to chunks, so the hicpro output is a little different.

  1. In bwt2, there is '.bwt2pairs.RSstat' for each chunk, but not for the whole data. The '.mpairstat' record the same thing though.
  2. In data, there are '.DEPairs','DumpPairs','.SCPairs','.SinglePairs','.validPairs','.RSstat' for each chunk, but not for the whole data.

hichipper runs fine with test data. When I run it for my own data, I encounter the following error. I do not know what that means. Is it because of the chunk thing?

Tue Mar 07 11:02:52 CST 2017:
['Rscript', u'output1/qcReport.R', '/Users/lab2116/Library/Python/2.7/lib/python/site-packages/hichipper', 'output1', '/Users/lab2116/Desktop/analysis/hichipper-master', '0.4.20', 'split']
/Users/lab2116/Desktop/analysis/hichipper-masterError: pandoc version 1.12.3 or higher is required and was not found (see the help page ?rmarkdown::pandoc_available).
Execution halted
Processing: split
Error in { : task 1 failed - "missing value where TRUE/FALSE needed"
Calls: %do% ->
Execution halted

PS: another question is about pandoc. I download the rstudio binary and try to install it, but it says can not install windows binary. I tried unzip the file and install the files in /mac, it does not work eaither.

Bedtools does not exist, but we need it

Hello,when I run hichipper,I met a problem that:Bedtools does not exist, but we need i. In our server , there are a lot numbers of bedtools of different versions . However , in ~/.bashrc file ,I define a environmental path to the newest version of bedtools. However , i failed to run hichipper.
why?
Thanks!

"Column index must be at most 0 if positive, not 1" error

Hello, I tried to run hichipper with a test dataset (JC020 below) and I got the following obscure error at the bottom. It seems to be related to the qcReport_make.Rmd script, but I can't pinpoint what is happening. When I run hichipper on the example datasets (e.g. one.yaml), it seems to work. The input was generated with HiC-Pro, and I deleted all the extraneous files in bowtie_results and hic_results to match the directory structure of the examples. Would greatly appreciate any insight into why this error is popping up and how one might go about fixing the input. Thanks!

hichipper --out JC020_out JC020.yaml --skip-diffloop

Thu Jun 01 10:39:00 PDT 2017: Starting hichipper pipeline v0.6.0
Thu Jun 01 10:39:00 PDT 2017: Parsing user parameters
Thu Jun 01 10:54:06 PDT 2017: Performing hichipper-modified background peak calling
Read 6590290 rows and 6 (of 6) columns from 0.271 GB file in 00:00:07
Read 65171550 rows and 4 (of 4) columns from 1.935 GB file in 00:00:23
Read 43535862 rows and 4 (of 4) columns from 1.292 GB file in 00:00:14
Thu Jun 01 11:07:10 PDT 2017: Modified background signal; performing peak calling
Thu Jun 01 11:07:10 PDT 2017: MACS2 TEXT OUTPUT INCOMING
INFO @ Thu, 01 Jun 2017 11:07:10: Read and build treatment bedGraph...
INFO @ Thu, 01 Jun 2017 11:08:23: Read and build control bedGraph...
INFO @ Thu, 01 Jun 2017 11:09:08: Build scoreTrackII...
INFO @ Thu, 01 Jun 2017 11:09:47: Calculate scores comparing treatment and control by 'qpois'...
INFO @ Thu, 01 Jun 2017 11:10:50: Write bedGraph of scores...
INFO @ Thu, 01 Jun 2017 11:11:47: Finished 'qpois'! Please check 'JC020_out/hichipper_qvalue.bdg.tmp'!
INFO @ Thu, 01 Jun 2017 11:11:47: Read and build bedGraph...
INFO @ Thu, 01 Jun 2017 11:12:08: Call peaks from bedGraph...
INFO @ Thu, 01 Jun 2017 11:12:09: Write peaks...
INFO @ Thu, 01 Jun 2017 11:12:09: Done
Read 6590290 rows and 6 (of 6) columns from 0.271 GB file in 00:00:08
Anchors removed due to excessive size (likely at ends of chromosomes): 6
Thu Jun 1 11:12:28 PDT 2017: Processing JC020
head: /Users/michael/Temporary/hichipper_input//Users/Michael/Temporary/hichipper_input/JC020/bowtie_results/bwt2/JC020/*pairstat: No such file or directory
Thu Jun 1 11:12:28 PDT 2017: Total_PETs=
cat: /Users/michael/Temporary/hichipper_input//Users/Michael/Temporary/hichipper_input/JC020/hic_results/data/JC020/Pairs: No such file or directory
Thu Jun 1 11:12:28 PDT 2017: Mapped_unique_quality_pairs=0
cat: /Users/michael/Temporary/hichipper_input//Users/Michael/Temporary/hichipper_input/JC020/hic_results/data/JC020/
_allValidPairs: No such file or directory
Thu Jun 1 11:12:28 PDT 2017: Mapped_unique_quality_valid_pairs=0
Thu Jun 1 11:12:28 PDT 2017: Intersecting PETs with anchors
Thu Jun 1 11:12:28 PDT 2017: Finished the anchor merging.
cat: /Users/michael/Temporary/hichipper_input//Users/Michael/Temporary/hichipper_input/JC020/hic_results/data/JC020/*Pairs: No such file or directory
cat: /Users/michael/Temporary/hichipper_input//Users/Michael/Temporary/hichipper_input/JC020/hic_results/data/JC020/Pairs: No such file or directory
cat: /Users/michael/Temporary/hichipper_input//Users/Michael/Temporary/hichipper_input/JC020/hic_results/data/JC020/
_allValidPairs: No such file or directory
Thu Jun 1 11:12:28 PDT 2017: Intrachromosomal_valid_small=0
Thu Jun 1 11:12:28 PDT 2017: Intrachromosomal_valid_med=0
Thu Jun 1 11:12:28 PDT 2017: Intrachromosomal_valid_large=0
Thu Jun 1 11:12:28 PDT 2017: Total number of anchors used: 311
Thu Jun 1 11:12:28 PDT 2017: Total number of reads in anchors: 0
Thu Jun 1 11:12:28 PDT 2017: Mapped_unique_intra_quality_anchor=0
Thu Jun 1 11:12:28 PDT 2017: Mapped_unique_intra_quality_anchor_small=0
Thu Jun 1 11:12:28 PDT 2017: Mapped_unique_intra_quality_anchor_med=0
Thu Jun 1 11:12:28 PDT 2017: Mapped_unique_intra_quality_anchor_large=0
Thu Jun 1 11:12:28 PDT 2017: Loop_PETs=
Thu Jun 01 11:12:28 PDT 2017:
['Rscript', u'JC020_out/qcReport.R', '/Library/Python/2.7/site-packages/hichipper', 'JC020_out', '/Users/michael/Temporary/hichipper_input', '0.6.0', 'JC020']
/Users/michael/Temporary/hichipper_input

processing file: qcReport_make.Rmd
|... | 5%
inline R code fragments

|....... | 11%
label: unnamed-chunk-1 (with options)
List of 3
$ echo : logi FALSE
$ message: logi TRUE
$ warning: logi TRUE

|.......... | 16%
inline R code fragments

|.............. | 21%
label: unnamed-chunk-2 (with options)
List of 3
$ echo : logi FALSE
$ message: logi FALSE
$ warning: logi FALSE

|................. | 26%
ordinary text without R code

|..................... | 32%
label: unnamed-chunk-3 (with options)
List of 4
$ echo : logi FALSE
$ message: logi FALSE
$ warning: logi FALSE
$ results: chr "asis"

|........................ | 37%
ordinary text without R code

|........................... | 42%
label: unnamed-chunk-4 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

|............................... | 47%
ordinary text without R code

|.................................. | 53%
label: unnamed-chunk-5 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

|...................................... | 58%
ordinary text without R code

|......................................... | 63%
label: unnamed-chunk-6 (with options)
List of 6
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ results : chr "asis"
$ out.width: chr "\textwidth"
$ fig.width: num 7

Quitting from lines 166-189 (qcReport_make.Rmd)
Error in { :
task 1 failed - "Column index must be at most 0 if positive, not 1"
Calls: ... withCallingHandlers -> withVisible -> eval -> eval -> %do% ->

Hichipper with HiCUP input

Hello Caleb,

We are interested in running hichipper on our HiChIP data. However, your pipelines (Hi-C, capture HiC and HiChIP) are based on HiCUP which results in a bam that contains all the Valid reads. Is it possible to use hichipper with HiCUP?
If you can think of a quick fix then I can help edit the code for our internal version to include hichipper in our institute's workflow.

Thanks!
Rola

About output file of *****.gz.txt

Hi, I met a new problem.So sorry to bother you many times.
This time, I am confused about *****.gz.txt which can be loaded to UCSC.However,I cant't see the interaction but a single line of each PET(there is no line between PETs).
And the file is about :for example:
chr1 193775080 193780032 chr1:193403318-193409437,9 289586 .
Is that means PET "chr1 193775080 193780032" and PET "chr1:193403318-193409437" are interacted.And what "9" and "289586" means?
Thank you!

R dependencies are hard to figure out

Hi

I run into the problem, that the qc_report fails because of missing R libraries. It would be great if you would write all the dependencies in the README. As is, a user must figure out, he can select the '--keep-temp-files' so that the R scripts are deleted to then inspect what libraries are required.

For other users: My R installation was missing readr, DT and networkD3

Best
Michael

Can't get right results for simple usage example

I tried to check if hichipper installed sucessfully by running simple usage example. My command is "hichipper --out output1 yaml/one.yaml", but it looks like existing some problems. The output is as following.
Tue May 23 12:40:26 CST 2017: Starting hichipper pipeline v0.6.0
Tue May 23 12:40:26 CST 2017: Parsing user parameters
Anchors removed due to excessive size (likely at ends of chromosomes): 0
Tue May 23 12:40:29 CST 2017: Processing dSRR3467177
Tue May 23 12:40:29 CST 2017: Total_PETs=8658944
Tue May 23 12:40:29 CST 2017: Mapped_unique_quality_pairs=156192
Tue May 23 12:40:29 CST 2017: Mapped_unique_quality_valid_pairs=56517
Tue May 23 12:40:29 CST 2017: Intersecting PETs with anchors
Tue May 23 12:40:29 CST 2017: Finished the anchor merging.
Tue May 23 12:40:33 CST 2017: Intrachromosomal_valid_small=8811
Tue May 23 12:40:33 CST 2017: Intrachromosomal_valid_med=21314
Tue May 23 12:40:33 CST 2017: Intrachromosomal_valid_large=3640
Tue May 23 12:40:33 CST 2017: Total number of anchors used: 926
Tue May 23 12:40:33 CST 2017: Total number of reads in anchors: 27298
Tue May 23 12:40:33 CST 2017: Mapped_unique_intra_quality_anchor=1138
Tue May 23 12:40:33 CST 2017: Mapped_unique_intra_quality_anchor_small=763
Tue May 23 12:40:33 CST 2017: Mapped_unique_intra_quality_anchor_med=352
Tue May 23 12:40:33 CST 2017: Mapped_unique_intra_quality_anchor_large=23
Tue May 23 12:40:33 CST 2017: Loop_PETs=352
Tue May 23 12:40:33 CST 2017:
['Rscript', u'output1/qcReport.R', '/home/students/pinlv/Luz/python/lib/python2.7/site-packages/hichipper', 'output1', '/home/students/pinlv/Luz/hichipper/hichipper/tests', '0.6.0', 'dSRR3467177']
/home/students/pinlv/Luz/hichipper/hichipper/tests

processing file: qcReport_make.Rmd
|... | 5%
inline R code fragments

|....... | 11%
label: unnamed-chunk-1 (with options)
List of 3
$ echo : logi FALSE
$ message: logi TRUE
$ warning: logi TRUE

|.......... | 16%
inline R code fragments

|.............. | 21%
label: unnamed-chunk-2 (with options)
List of 3
$ echo : logi FALSE
$ message: logi FALSE
$ warning: logi FALSE

|................. | 26%
ordinary text without R code

|..................... | 32%
label: unnamed-chunk-3 (with options)
List of 4
$ echo : logi FALSE
$ message: logi FALSE
$ warning: logi FALSE
$ results: chr "asis"

|........................ | 37%
ordinary text without R code

|........................... | 42%
label: unnamed-chunk-4 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

|............................... | 47%
ordinary text without R code

|.................................. | 53%
label: unnamed-chunk-5 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

|...................................... | 58%
ordinary text without R code

|......................................... | 63%
label: unnamed-chunk-6 (with options)
List of 6
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ results : chr "asis"
$ out.width: chr "\textwidth"
$ fig.width: num 7

|............................................ | 68%
ordinary text without R code

|................................................ | 74%
label: unnamed-chunk-7 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

Using count as value column: use value.var to override.
|................................................... | 79%
ordinary text without R code

|....................................................... | 84%
label: unnamed-chunk-8 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

|.......................................................... | 89%
ordinary text without R code

|.............................................................. | 95%
label: unnamed-chunk-9 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

|.................................................................| 100%
ordinary text without R code

output file: qcReport_make.knit.md

/home/students/pinlv/bin/pandoc +RTS -K512m -RTS qcReport_make.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output output1.hichipper.qcreport.html --smart --email-obfuscation none --self-contained --standalone --section-divs --template /home/students/pinlv/R/x86_64-pc-linux-gnu-library/3.3/rmarkdown/rmd/h/default.html --highlight-style pygments --css /home/students/pinlv/R/x86_64-pc-linux-gnu-library/3.3/rmarkdown/rmarkdown/templates/html_vignette/resources/vignette.css --include-in-header /tmp/RtmpDfS9nh/rmarkdown-str85202723e301.html --mathjax --variable 'mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'

Output created: output1.hichipper.qcreport.html
Warning messages:
1: Transformation introduced infinite values in continuous x-axis
2: Removed 751 rows containing non-finite values (stat_bin).
Processing: dSRR3467177
Error in { : task 1 failed - "missing value where TRUE/FALSE needed"
Calls: %do% ->
In addition: There were 35 warnings (use warnings() to see them)
Execution halted

I think there are problems with calling interaction, but I don't know how to solve them. Could you help me to deal with these bugs? Thanks so much!
Luzhang Ji

Absolute file path

Currently there's an error thrown when users supply the absolute file path to a bam

Provided hg19 restriction fragment file includes 0bp entry

Maybe not an actual problem when using hichipper, but I was just using the provided hg19 MboI restriction fragment files and found something weird in it. It includes a 0-bp entry on chrM.

> wget -O hg19_MboI_resfrag.bed.gz https://github.com/aryeelab/hichipper/blob/master/RestrictionFragmentFiles/hg19_MboI_resfrag.bed.gz?raw=true
> gzip -d hg19_MboI_resfrag.bed.gz
> bedtools sort -i hg19_MboI_resfrag.bed > hg19_MboI_resfrag.sorted.bed
Error: malformed BED entry at line 7127610. Start was greater than end. Exiting.

In that line:

chrM    0       0
chrM    0       741
chrM    741     952
chrM    952     1228
chrM    1228    2350

With HiC-Pro 2.10.0 digest_genome.py:

chrM    0       741     HIC_chrM_1      0       +
chrM    741     952     HIC_chrM_2      0       +
chrM    952     1228    HIC_chrM_3      0       +
chrM    1228    2350    HIC_chrM_4      0       +

Interestingly, this is not true for hg38.
Were the files produced with HiC-Pro and if so, with which version?

Warning messages in QC report generation

I installed hichipper on our cluster and ran the example as follows:

cp -r /path/to/hichipper/tests .
cd tests
hichipper --out output1 yaml/one.yaml

and the run appears to complete successfully, but the last thing printed is this:

Output created: output1.hichipper.qcreport.html
Warning messages:
1: Transformation introduced infinite values in continuous x-axis
2: Removed 13419 rows containing non-finite values (stat_bin).
Processing: d1

Is that an expected/ignorable warning for this example run?

Rendering HTML reports

Just recently, running the QC report both on my mac and Erisone doesn't yield the DT or D3network content; unclear why... Likely an issue with Rmarkdown/pandoc

ERROR: Missing hic_results or bowtie_results files for sample

hello, I meet a error when I used hichipper.

This is my hichipper Command Line and error:

hichipper --out hichipper_res hichipper_config.yaml
Thu Nov 02 22:05:39 CST 2017: Starting hichipper pipeline v0.6.1
Thu Nov 02 22:05:39 CST 2017: Parsing user parameters
ERROR: Missing hic_results or bowtie_results files for sample; either exclude the sample or manage the file architecture/input

and this is my hichipper_config.yaml:

peaks:
- ChIP-Seq_human_SRR502135_36_SMC3_bowtie2_merged_hichipper_peaks.narrowPeak
resfrags:
- MboI_resfrag_hg19.bed
hicpro_output:
- hicpro_simplify

and this is hicpro_simplify file:

hicpro_simplify/bowtie_results/bwt2/sample:
SRR3467175_hg19.bwt2pairs.pairstat

hicpro_simplify/hic_results/data/sample:
SRR3467175_allValidPairs             SRR3467175_hg19.bwt2pairs.REPairs  SRR3467175_hg19.bwt2pairs.SinglePairs
SRR3467175_hg19.bwt2pairs.DEPairs    SRR3467175_hg19.bwt2pairs.RSstat   SRR3467175_hg19.bwt2pairs.validPairs
SRR3467175_hg19.bwt2pairs.DumpPairs  SRR3467175_hg19.bwt2pairs.SCPairs

I checked my directory stucture with your manual, but I don't know where is wrong.
could you help me to solve it?
Thank you!

could not find function "endsWith"

I was trying out the installation guide.
Installation seems fine.
When I was trying this command,
"/home/wlwtan/.local/bin/hichipper --out output1 yaml/one.yaml",
it failed with these following error:
Thu Aug 30 20:06:54 SGT 2018: Starting hichipper pipeline v0.7.3 Thu Aug 30 20:06:54 SGT 2018: Parsing user parameters Thu Aug 30 20:06:54 SGT 2018: .yaml file detected **Error: could not find function "endsWith"** Execution halted Thu Aug 30 20:07:00 SGT 2018: Processing d1 Thu Aug 30 20:07:01 SGT 2018: Total_PETs=164332898 Thu Aug 30 20:07:14 SGT 2018: Mapped_unique_quality_pairs=1888170 Thu Aug 30 20:07:14 SGT 2018: Mapped_unique_quality_valid_pairs=627403 Thu Aug 30 20:07:14 SGT 2018: Intersecting PETs with anchors **Error: The requested file (output1/userSuppliedPeaks.bed.tmp_pad.bed.tmprf.tmp) could not be opened. Error message: (No such file or directory). Exiting!** /home/wlwtan/.local/lib/python2.7/site-packages/hichipper/interactionsCall.sh: line 36: --keep-temp-files: command not found Thu Aug 30 20:07:15 SGT 2018: Something went wrong in determining peaks for anchor inference; rerun with the flag to debug. **ERROR: something failed at the individual sample level; check the .log file for more info**
Not sure what have gone wrong.

Thank you.

Error message regarding 'x' values

Processing: DB-1-data
Error in { : task 1 failed - "need at least four unique 'x' values"
Calls: %do% ->

Would you happen to know what is causing this issue?

.yaml round trip

Considered my options for a while. I think that the best approach is to do the .yaml round trip and work with an external python script to delineate old and new hichipper versions based on how rigid I made the HiC-Pro integration

question about error message

Hello! I had a question regrading running hichipper

Fri Jun 22 16:00:24 EDT 2018: Starting hichipper pipeline v0.7.3
Fri Jun 22 16:00:24 EDT 2018: Parsing user parameters
Fri Jun 22 16:00:24 EDT 2018: .yaml file detected
ls: cannot access hicpro/bowtie_results/bwt2/mcd1_rep1/*.pairstat: No such file or directory

The name of my files, I have show below. It seems I don't have the pairstat file, but this is the output I obtained from the HiCPro. Can I use these files? The output contained:

In a directory called mcd1_rep1:
DB-1_R1_001_hg19.bwt2glob.bam
DB-1_R2_001_hg19.bwt2glob.bam
DB-1_R1_001_hg19.bwt2glob.unmap_trimmed.fastq
DB-1_R2_001_hg19.bwt2glob.unmap.fastq

Am I missing files?

empty interaction

Hi,
I used the following command (when use test folder it is ok with the output file)
hichipper --out TKO.loop TKO.yaml --skip-resfrag-pad
but for my samples, it output empty files (no error reports):
0 Oct 3 20:04 fq.filt.intra.loop_counts.bedpe
0 Oct 3 20:04 fq.inter.loop_counts.bedpe
0 Oct 3 20:04 fq.intra.loop_counts.bedpe
492 Oct 3 20:04 fq.stat
2001 Oct 3 20:04 TKO.E12.loop.hichipper.log

log:
Tue Oct 03 19:17:21 CDT 2017: Starting hichipper pipeline v0.7.0
Tue Oct 03 19:17:21 CDT 2017: Parsing user parameters
Tue Oct 3 19:17:21 CDT 2017: Processing fq
Tue Oct 3 19:17:21 CDT 2017: Total_PETs=699057358
Tue Oct 3 19:19:28 CDT 2017: Mapped_unique_quality_pairs=565946889
Tue Oct 3 19:19:45 CDT 2017: Mapped_unique_quality_valid_pairs=167545325
Tue Oct 3 19:19:45 CDT 2017: Intersecting PETs with anchors
Tue Oct 3 19:19:45 CDT 2017: Finished the anchor merging.
Tue Oct 3 20:06:12 CDT 2017: Intrachromosomal_valid_small=0
Tue Oct 3 20:06:12 CDT 2017: Intrachromosomal_valid_med=0
Tue Oct 3 20:06:12 CDT 2017: Intrachromosomal_valid_large=0
Tue Oct 3 20:06:12 CDT 2017: Total number of anchors used: 164465
Tue Oct 3 20:06:12 CDT 2017: Total number of reads in anchors: 152747515
Tue Oct 3 20:06:12 CDT 2017: Mapped_unique_intra_quality_anchor=0
Tue Oct 3 20:06:12 CDT 2017: Mapped_unique_intra_quality_anchor_small=0
Tue Oct 3 20:06:12 CDT 2017: Mapped_unique_intra_quality_anchor_med=0
Tue Oct 3 20:06:12 CDT 2017: Mapped_unique_intra_quality_anchor_large=0
Tue Oct 3 20:06:12 CDT 2017: Loop_PETs=

Where is the problem?

Thanks

If there any output file about peak calling

Hi, I have use this software well . However , I can't find any files out abut peaks since I use .bam file to call peak with MACS2.
Can I obtain it by some extra parameter?

chipseq input

Hi I had a question about setting up the yaml file.

If I have two conditions in my hic-pro output folder let's say WT and Mutant, how do I make sure the chipseq files I indicate in the yaml will match the correct output in the HiC-pro output folder.

i.e.

peaks:

  • chipseq/WT.bed
  • chipseq/Mutant.bed
    resfrags:
  • /ifs/home/sb5169/hg38/MboI_resfrag_hg38.bed
    hicpro_output:
  • /ifs/home/sb5169/PL_HiChIP2/Pooled_output

Facile integration to WashU genome browser

Since I don't want to keep maintaining DNAlandscapeR at production-level quality...

  1. Need to support loop calls in a supported WashU genome browser format:
    http://wiki.wubrowse.org/Long-range
    Seemingly, the best way to do this would be to compress and index the existing .bedpe files using a native python module (e.g. https://pypi.python.org/pypi/pytabix/0.1) and including it as the default output.

  2. Need to set up a json when files are hosted on a public online repo (Amazon, DropBox, etc.)
    http://epigenomegateway.wustl.edu/support/guide/data_management.html

The .json or loops files (when properly formatted) can be imported here: http://epigenomegateway.wustl.edu/browser/

Perhaps in a .log -style file, we can write the .json bits associated with the output to make this faster. Or just show something in the README for how to write one of these quickly.

@martinaryee any thoughts on any of this?

Add option to keep temporary files

Thanks for developing the software!
Would be nice to have an option not to remove temporary files - especially the R scripts.
This would help debugging during the first runs.

Error in .Call2("NCList_find_overlaps_in_groups", start(q), end(q), q_space

I run the test data by (nohup hichipper --out output2 yaml/example_EACH_ALL.yaml &) ,but get the below erros:
Wed May 30 10:10:39 CST 2018: Starting hichipper pipeline v0.7.0
Wed May 30 10:10:39 CST 2018: Parsing user parameters
Wed May 30 10:10:47 CST 2018: Performing hichipper-modified background peak calling
Error in .Call2("NCList_find_overlaps_in_groups", start(q), end(q), q_space, :
when 'type' is "any", at least one of 'maxgap' and 'minoverlap' must be set to its default value
Calls: computeRatioEtc ... findOverlaps_GNCList -> -> .Call2 -> .Call
Execution halted

Rscript requirementsInstall.R fails with error in install.packages

Error:

Error in contrib.url(repos, type) :
  trying to use CRAN without setting a mirror
Calls: install_pkgs -> install.packages -> grep -> contrib.url
Execution halted

Fix: Added the parameter "repos" to install.packages on line 3
if (length(new.pkg)) install.packages(new.pkg, dependencies = TRUE, repos="https://cloud.r-project.org")

About the *.filt.intra.loop_counts.bedpe format

Hi there,

I got the *.filt.intra.loop_counts.bedpe, but what the third column "1" mean ?
Is the third column mean PET number that support this loop? Should I filter again based on third column? How to filter?

How can I get the p/fdr value for each loop?

Thanks

Best

Jia Li

chr10,100015532,100016705 chr10,100118360,100119539 1
chr10,100015532,100016705 chr10,100129996,100131143 1
chr10,100015532,100016705 chr10,100153694,100154869 1
chr10,100027364,100028596 chr10,100038190,100039338 1
chr10,100032538,100033776 chr10,100038190,100039338 1
chr10,100032538,100033776 chr10,100068721,100069868 1
chr10,100032538,100033776 chr10,100180125,100181277 1
chr10,100032538,100033776 chr10,100603185,100604332 1
chr10,100038190,100039338 chr10,100168859,100170006 1
chr10,100038190,100039338 chr10,100431107,100432285 1

the difference between *.interaction.txt.gz and *.filt.intra.loop_counts.bedpe

Hi,

Hichipper documentation said *.interaction.txt.gz and *.filt.intra.loop_counts.bedpe are identical except in presentation, so I think their number of rows are the same. But in my hichipper output files, the rows number are different, *.interaction.txt.gz has more rows than *.filt.intra.loop_counts.bedpe dose.

Is there something wrong in using the hichipper?

Thanks in advance

ERROR: macs2 peak calling was not successful

Hello! I'm trying to test hichipper in my programming environment by running the following lines from the /tests/README.md:

hichipper call --out outputCallBAM --restriction-frags ../RestrictionFragmentFiles/hg19_MboI_resfrag.bed.gz --peaks chipseq/GM12878_SMC3_ChIPSeq_chr22.narrowPeak --skip-resfrag-pad --basic-qc --skip-diffloop --input-bam call_inputs/HiCUP_chr22.bam

hichipper --out combinedall --skip-diffloop yaml/example_COMBINED_ALL.yaml
hichipper --out combinedself --skip-diffloop yaml/example_COMBINED_SELF.yaml

First error:
[u'outputCallvi/userSuppliedPeaks.bed.tmp_pad.bed.tmp']
Not fully supported yet!
What does this mean and what can I do to fix it?

Second error (applies to both the combined lines):
AttributeError: 'Namespace' object has no attribute 'maxgap'
ERROR: macs2 peak calling was not successful; make sure macs2 is in the environment and that *Pairs files exist for samples. Finally, note that only the narrowPeak file is being considered here, which may be a problem if you tried to do broad peak calling.
MACS2 is in my PATH so I'm not sure how to fix this one either.

I appreciate any help I can get!
-Andrea

--make-ucsc flag error

To whom it may concern,

I'm receiving the following message when I use the flag --make-ucsc.

Fri Aug 17 12:45:29 CEST 2018: Starting hichipper pipeline v0.7.3 Fri Aug 17 12:45:29 CEST 2018: Parsing user parameters Fri Aug 17 12:45:29 CEST 2018: .yaml file detected ERROR: Could not import any samples from the user specification; check flags, logs and .yaml configuration; quitting

-I have htslib and tabix (together with samtools) installed.

-My .yaml file works without the --make-ucsc flag

`
peaks:

  • EACH,ALL
    resfrags:
  • /Users/raphael/Desktop/Bioinformatic/test/hg19_MboI_resfrag.bed
    hicpro_output:
  • /Users/raphael/Desktop/Bioinformatic/test/HiChIP_HiCPro_output

`

  • I'm running one sample only

Any ideas what could the source of the error?

Many thanks

About Macs2 parameter

Hi, I have use this software well . However , I failed to pass parameter to MACS2 . For example ,I just want to change "p" value from 0.01 to 0.0001 as " --macs2-string "-q 0.0001" " or " --macs2-string -q 0.0001"
But I got Error: Got unexpected extra argument (0.0001)
Why?
What can I do ?
Thanks!

Use HiC-Pro allValidPairs as input

It would be great to be able to specify a hicpro allValidPairs file as input for hichipper instead of the full hicpro output dir. The other inputs would probably be a restriction fragment bed file and possible a peak set.

This feature would make it easier to put hichipper into a pipeline that runs hicpro as a first step where we only retain a few key output files (e.g. allValidPairs).

memory issues

Current implementation of background correction requires significant RAM to compute

Data test exception

I'm trying to run the example data test. But I got the following problem.

screen shot 2018-05-14 at 10 30 37 pm

Which would be the problem ?
Thanks in advance!

About html report

Hi,thank you for such program.
But I am confused about html report.
1)Total_PETS are different from the file I offer (I only offer _allValidPairs and with 240661910 valid pairs(I just use "wc -l " to check).However ,the html file said this file is 336,357,341 valid pairs.
And I check the "bwt2pairs.RSstat" ,but sum all type of pairs and which don't equal to 336,357,341

2)such html file also provide "Mapped_unique_quality", which is "2.8933" from Proportional Read Stats. But such step I already did in Hic-Pro,therefore it should be "0" of "Mapped_unique_quality"

why?
Thank you!

*_allValidPairs: No such file or directory

HiC-Pro does not (or no longer) outputs files that will match *_allValidPairs glob, rather the files are named <Sample>.allValidPairs. It's easy enough to make symlinks that match your glob pattern, but probably an easy fix on your end in the hichipper code to fix this.

thanks for the great package!

v0.4.4 < version < v0.5.6 -- duplicates

Need to parse the *_allValidPairs file (PCR de-duplicated) rather than the *validPairs

interactionsCall.sh line 24; simple fix.

cli.py line 109 -- need to verify the *_allValidPairs

Only affects the versions listed in the title. Will make this version 0.6.0

Error in { : task 1 failed - "need at least four unique 'x' values"

Hi, I am having an issue using hichipper regarding a specific anchor configuration. Interestingly, this only occurs when I am running hichipper using EACH,SELF to identify % in Loops. Using the same dataset, everything runs completely fine when supplying my own anchors from a separate ChIP-seq experiment, as well as calling peaks internally using COMBINED,ALL/COMBINED-SELF/EACH,ALL. The EACH,SELF command still runs and gives an output, but this results in extremely low peak calling in one of the samples, whereas all the other methods identify high % in Loops. The output from the error is attached below:

hichipper --out Control_HDAC_SMC3_HiChIPPeaks_EACH-SELF HiChIPPer.yaml
Thu Apr 19 12:18:40 CDT 2018: Starting hichipper pipeline v0.7.1
Thu Apr 19 12:18:40 CDT 2018: Parsing user parameters
Thu Apr 19 12:19:29 CDT 2018: Performing hichipper-modified background peak calling
Read 7227576 rows and 6 (of 6) columns from 0.297 GB file in 00:00:09
Thu Apr 19 12:21:13 CDT 2018: Modified background signal; performing peak calling
Thu Apr 19 12:21:13 CDT 2018: MACS2 TEXT OUTPUT INCOMING
INFO @ Thu, 19 Apr 2018 12:21:14: Read and build treatment bedGraph...
INFO @ Thu, 19 Apr 2018 12:21:18: Read and build control bedGraph...
INFO @ Thu, 19 Apr 2018 12:21:21: Build scoreTrackII...
INFO @ Thu, 19 Apr 2018 12:21:24: Calculate scores comparing treatment and control by 'qpois'...
INFO @ Thu, 19 Apr 2018 12:21:27: Write bedGraph of scores...
INFO @ Thu, 19 Apr 2018 12:21:36: Finished 'qpois'! Please check 'Control_HDAC_SMC3_HiChIPPeaks_EACH-SELF/hichipper_qvalue.bdg.tmp'!
INFO @ Thu, 19 Apr 2018 12:21:36: Read and build bedGraph...
INFO @ Thu, 19 Apr 2018 12:21:41: Call peaks from bedGraph...
INFO @ Thu, 19 Apr 2018 12:21:41: Write peaks...
INFO @ Thu, 19 Apr 2018 12:21:41: Done
Thu Apr 19 12:33:54 CDT 2018: Performing hichipper-modified background peak calling
Read 7227576 rows and 6 (of 6) columns from 0.297 GB file in 00:00:09
Read 66417984 rows and 4 (of 4) columns from 1.984 GB file in 00:00:19
Read 36695775 rows and 4 (of 4) columns from 1.095 GB file in 00:00:10
Thu Apr 19 12:45:56 CDT 2018: Modified background signal; performing peak calling
Thu Apr 19 12:45:56 CDT 2018: MACS2 TEXT OUTPUT INCOMING
INFO @ Thu, 19 Apr 2018 12:45:57: Read and build treatment bedGraph...
INFO @ Thu, 19 Apr 2018 12:46:59: Read and build control bedGraph...
INFO @ Thu, 19 Apr 2018 12:47:32: Build scoreTrackII...
INFO @ Thu, 19 Apr 2018 12:48:02: Calculate scores comparing treatment and control by 'qpois'...
INFO @ Thu, 19 Apr 2018 12:48:52: Write bedGraph of scores...
INFO @ Thu, 19 Apr 2018 12:50:01: Finished 'qpois'! Please check 'Control_HDAC_SMC3_HiChIPPeaks_EACH-SELF/hichipper_qvalue.bdg.tmp'!
INFO @ Thu, 19 Apr 2018 12:50:03: Read and build bedGraph...
INFO @ Thu, 19 Apr 2018 12:50:35: Call peaks from bedGraph...
INFO @ Thu, 19 Apr 2018 12:50:37: Write peaks...
INFO @ Thu, 19 Apr 2018 12:50:38: Done
Read 7227576 rows and 6 (of 6) columns from 0.297 GB file in 00:00:09
Anchors removed due to excessive size (likely at ends of chromosomes): 15
Read 7227576 rows and 6 (of 6) columns from 0.297 GB file in 00:00:09
Anchors removed due to excessive size (likely at ends of chromosomes): 5
Thu Apr 19 12:51:25 CDT 2018: Processing Control_SMC3
Thu Apr 19 12:51:25 CDT 2018: Total_PETs=48274000
Thu Apr 19 12:53:20 CDT 2018: Mapped_unique_quality_pairs=70637958
Thu Apr 19 12:54:06 CDT 2018: Mapped_unique_quality_valid_pairs=33255507
Thu Apr 19 12:54:06 CDT 2018: Intersecting PETs with anchors
Thu Apr 19 12:54:12 CDT 2018: Finished the anchor merging.
Thu Apr 19 13:14:28 CDT 2018: Intrachromosomal_valid_small=3680202
Thu Apr 19 13:15:50 CDT 2018: Intrachromosomal_valid_med=17558449
Thu Apr 19 13:17:05 CDT 2018: Intrachromosomal_valid_large=5866650
Thu Apr 19 13:17:05 CDT 2018: Total number of anchors used: 631
Thu Apr 19 13:17:05 CDT 2018: Total number of reads in anchors: 262409
Thu Apr 19 13:21:05 CDT 2018: Mapped_unique_intra_quality_anchor=4647
Thu Apr 19 13:21:05 CDT 2018: Mapped_unique_intra_quality_anchor_small=2357
Thu Apr 19 13:21:05 CDT 2018: Mapped_unique_intra_quality_anchor_med=2286
Thu Apr 19 13:21:05 CDT 2018: Mapped_unique_intra_quality_anchor_large=4
Thu Apr 19 13:21:05 CDT 2018: Loop_PETs=2286
Thu Apr 19 13:21:05 CDT 2018: Processing HDAC_SMC3
Thu Apr 19 13:21:05 CDT 2018: Total_PETs=111263839
Thu Apr 19 13:24:30 CDT 2018: Mapped_unique_quality_pairs=125986837
Thu Apr 19 13:25:27 CDT 2018: Mapped_unique_quality_valid_pairs=33680149
Thu Apr 19 13:25:27 CDT 2018: Intersecting PETs with anchors
Thu Apr 19 13:25:32 CDT 2018: Finished the anchor merging.
Thu Apr 19 13:57:17 CDT 2018: Intrachromosomal_valid_small=16987842
Thu Apr 19 13:58:37 CDT 2018: Intrachromosomal_valid_med=9413688
Thu Apr 19 13:59:52 CDT 2018: Intrachromosomal_valid_large=2993967
Thu Apr 19 13:59:52 CDT 2018: Total number of anchors used: 78069
Thu Apr 19 13:59:52 CDT 2018: Total number of reads in anchors: 38007868
Thu Apr 19 14:05:35 CDT 2018: Mapped_unique_intra_quality_anchor=2128503
Thu Apr 19 14:05:35 CDT 2018: Mapped_unique_intra_quality_anchor_small=1751571
Thu Apr 19 14:05:35 CDT 2018: Mapped_unique_intra_quality_anchor_med=305072
Thu Apr 19 14:05:35 CDT 2018: Mapped_unique_intra_quality_anchor_large=71861
Thu Apr 19 14:05:39 CDT 2018: Loop_PETs=305072
Thu Apr 19 14:05:39 CDT 2018:
['Rscript', u'Control_HDAC_SMC3_HiChIPPeaks_EACH-SELF/qcReport.R', '/Library/Python/2.7/site-packages/hichipper', 'Control_HDAC_SMC3_HiChIPPeaks_EACH-SELF', '/Volumes/Terranova2/HDAC_2_3-25-16/HiC_Pro_Test_3_03-16-18', '0.7.1', 'Control_SMC3 HDAC_SMC3']
/Volumes/Terranova2/HDAC_2_3-25-16/HiC_Pro_Test_3_03-16-18Using count as value column: use value.var to override.
Processing: Control_SMC3HDAC_SMC3
Error in { : task 1 failed - "need at least four unique 'x' values"
Calls: %do% ->
Execution halted

Thanks for you help!!

Error in { : task 1 failed - "need at least four unique 'x' values" ; Calls: %do% -> <Anonymous>

I'm receiving the following error and I have no idea now where it is coming from

Error in { : task 1 failed - "need at least four unique 'x' values"
Calls: %do% -> <Anonymous>
Execution halted

I'm running the following command:

hichipper --out HiChIPper_full --make-ucsc config.yaml

with this .yaml:

peaks:
- COMBINED,SELF
resfrags:
- DpnII_resfrag_hg19.bed
hicpro_output:
- HiCPro_analysis

This is the stderr:

home:~/Desktop/Software/HiChipper$ hichipper --out HiChIPper_full --make-ucsc config.yaml
Thu Oct 11 18:21:07 CEST 2018: Starting hichipper pipeline v0.7.3
Thu Oct 11 18:21:07 CEST 2018: Parsing user parameters
Thu Oct 11 18:21:07 CEST 2018: .yaml file detected
Thu Oct 11 18:21:11 CEST 2018: Performing hichipper-modified background peak calling
Thu Oct 11 18:23:39 CEST 2018: Modified background signal; performing peak calling
Thu Oct 11 18:23:39 CEST 2018: MACS2 TEXT OUTPUT INCOMING
INFO  @ Thu, 11 Oct 2018 18:23:40: Read and build treatment bedGraph... 
INFO  @ Thu, 11 Oct 2018 18:23:40: Read and build control bedGraph... 
INFO  @ Thu, 11 Oct 2018 18:23:40: Build scoreTrackII... 
INFO  @ Thu, 11 Oct 2018 18:23:41: Calculate scores comparing treatment and control by 'qpois'... 
INFO  @ Thu, 11 Oct 2018 18:23:41: Write bedGraph of scores... 
INFO  @ Thu, 11 Oct 2018 18:23:42: Finished 'qpois'! Please check 'HiChIPper_full/hichipper_qvalue.bdg.tmp'! 
INFO  @ Thu, 11 Oct 2018 18:23:42: Read and build bedGraph... 
INFO  @ Thu, 11 Oct 2018 18:23:42: Call peaks from bedGraph... 
INFO  @ Thu, 11 Oct 2018 18:23:42: Write peaks... 
INFO  @ Thu, 11 Oct 2018 18:23:42: Done 
Anchors removed due to excessive size (likely at ends of chromosomes): 3 
Do 11. Okt 18:24:42 CEST 2018: Processing HiChIP
Do 11. Okt 18:24:42 CEST 2018: Total_PETs=820685
Do 11. Okt 18:24:42 CEST 2018: Mapped_unique_quality_pairs=473801
Do 11. Okt 18:24:42 CEST 2018: Mapped_unique_quality_valid_pairs=127230
Do 11. Okt 18:24:42 CEST 2018: Intersecting PETs with anchors
Do 11. Okt 18:24:42 CEST 2018: Finished the anchor merging.
Do 11. Okt 18:24:46 CEST 2018: Intrachromosomal_valid_small=33568
Do 11. Okt 18:24:46 CEST 2018: Intrachromosomal_valid_med=71004
Do 11. Okt 18:24:47 CEST 2018: Intrachromosomal_valid_large=10097
Do 11. Okt 18:24:47 CEST 2018: Total number of anchors used: 9635
Do 11. Okt 18:24:47 CEST 2018: Total number of reads in anchors: 98023
Do 11. Okt 18:24:47 CEST 2018: Mapped_unique_intra_quality_anchor=839
Do 11. Okt 18:24:47 CEST 2018: Mapped_unique_intra_quality_anchor_small=779
Do 11. Okt 18:24:47 CEST 2018: Mapped_unique_intra_quality_anchor_med=59
Do 11. Okt 18:24:47 CEST 2018: Mapped_unique_intra_quality_anchor_large=1
Do 11. Okt 18:24:47 CEST 2018: Creating UCSC Compatible files; make sure tabix and bgzip are available in the environment or this will not work.
Do 11. Okt 18:24:48 CEST 2018: Loop_PETs=59
Thu Oct 11 18:24:48 CEST 2018: 
['/usr/bin/Rscript', u'HiChIPper_full/qcReport.R', '/usr/local/lib/python2.7/dist-packages/hichipper', 'HiChIPper_full', '/home/Desktop/Software/HiChipper', '0.7.3', 'HiChIP']
/home/Desktop/Software/HiChipperUsing count as value column: use value.var to override.
Processing: HiChIP
Error in { : task 1 failed - "need at least four unique 'x' values"
Calls: %do% -> <Anonymous>
Execution halted

this is the output folder:

-rw-rw-r-- 1 home   2389 Okt 11 18:25 HiChIPper_full.hichipper.log
-rw-rw-r-- 1 home 377774 Okt 11 18:24 HiChIPper_full.hichipper.qcreport.html
-rw-rw-r-- 1 home 230555 Okt 11 18:24 HiChIP.anchors.bed
-rw-rw-r-- 1 home   2498 Okt 11 18:24 HiChIP.filt.intra.loop_counts.bedpe
-rw-rw-r-- 1 home   1780 Okt 11 18:24 HiChIP.interaction.txt.gz
-rw-rw-r-- 1 home   3513 Okt 11 18:24 HiChIP.interaction.txt.gz.tbi
-rw-rw-r-- 1 home    721 Okt 11 18:24 HiChIP.inter.loop_counts.bedpe
-rw-rw-r-- 1 home  24543 Okt 11 18:24 HiChIP.intra.loop_counts.bedpe
-rw-rw-r-- 1 home    499 Okt 11 18:24 HiChIP.stat

Thank you for your time,

Raphael

support absolute file paths

In the current implementation, the working directory is pre-pended to the files, which causes errors when the users supply absolute file paths.

Incomplete HTML output in test data and real data.

bash-4.1$ hichipper --out output_real yaml/one.yaml
Thu Oct 26 16:53:47 EDT 2017: Starting hichipper pipeline v0.7.0
Thu Oct 26 16:53:47 EDT 2017: Parsing user parameters
Anchors removed due to excessive size (likely at ends of chromosomes): 0
Thu Oct 26 16:53:50 EDT 2017: Processing d1
Thu Oct 26 16:53:50 EDT 2017: Total_PETs=164332898
Thu Oct 26 16:53:51 EDT 2017: Mapped_unique_quality_pairs=1888170
Thu Oct 26 16:53:51 EDT 2017: Mapped_unique_quality_valid_pairs=627403
Thu Oct 26 16:53:51 EDT 2017: Intersecting PETs with anchors
Thu Oct 26 16:53:51 EDT 2017: Finished the anchor merging.
Thu Oct 26 16:54:23 EDT 2017: Intrachromosomal_valid_small=163462
Thu Oct 26 16:54:24 EDT 2017: Intrachromosomal_valid_med=394048
Thu Oct 26 16:54:25 EDT 2017: Intrachromosomal_valid_large=69902
Thu Oct 26 16:54:25 EDT 2017: Total number of anchors used: 926
Thu Oct 26 16:54:25 EDT 2017: Total number of reads in anchors: 410414
Thu Oct 26 16:54:31 EDT 2017: Mapped_unique_intra_quality_anchor=20288
Thu Oct 26 16:54:31 EDT 2017: Mapped_unique_intra_quality_anchor_small=13632
Thu Oct 26 16:54:31 EDT 2017: Mapped_unique_intra_quality_anchor_med=6090
Thu Oct 26 16:54:31 EDT 2017: Mapped_unique_intra_quality_anchor_large=566
Thu Oct 26 16:54:31 EDT 2017: Loop_PETs=6090
Thu Oct 26 16:54:31 EDT 2017:
['Rscript', u'output_real/qcReport.R', '/usr/local/Anaconda/envs_app/hichipper/0.7.0/lib/python2.7/site-packages/hichipper', 'output_real', '/spin1/users/dalgleishjl/hichipper/tests', '0.7.0', 'd1']
/spin1/users/dalgleishjl/hichipper/tests

processing file: qcReport_make.Rmd
|... | 5%
inline R code fragments

|....... | 11%
label: unnamed-chunk-1 (with options)
List of 3
$ echo : logi FALSE
$ message: logi TRUE
$ warning: logi TRUE

|.......... | 16%
inline R code fragments

|.............. | 21%
label: unnamed-chunk-2 (with options)
List of 3
$ echo : logi FALSE
$ message: logi FALSE
$ warning: logi FALSE

|................. | 26%
ordinary text without R code

|..................... | 32%
label: unnamed-chunk-3 (with options)
List of 4
$ echo : logi FALSE
$ message: logi FALSE
$ warning: logi FALSE
$ results: chr "asis"

|........................ | 37%
ordinary text without R code

|........................... | 42%
label: unnamed-chunk-4 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

|............................... | 47%
ordinary text without R code

|.................................. | 53%
label: unnamed-chunk-5 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

|...................................... | 58%
ordinary text without R code

|......................................... | 63%
label: unnamed-chunk-6 (with options)
List of 6
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ results : chr "asis"
$ out.width: chr "\textwidth"
$ fig.width: num 7

|............................................ | 68%
ordinary text without R code

|................................................ | 74%
label: unnamed-chunk-7 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

Using count as value column: use value.var to override.
|................................................... | 79%
ordinary text without R code

|....................................................... | 84%
label: unnamed-chunk-8 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

|.......................................................... | 89%
ordinary text without R code

|.............................................................. | 95%
label: unnamed-chunk-9 (with options)
List of 4
$ echo : logi FALSE
$ message : logi FALSE
$ warning : logi FALSE
$ out.width: chr "\textwidth"

|.................................................................| 100%
ordinary text without R code

output file: qcReport_make.knit.md

/usr/local/apps/pandoc/1.15.0.6/bin/pandoc +RTS -K512m -RTS qcReport_make.utf8.md --to html --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash --output output_real.hichipper.qcreport.html --smart --email-obfuscation none --self-contained --standalone --section-divs --template /usr/local/apps/R/gcc_4.9.1/library/rmarkdown/rmd/h/default.html --highlight-style pygments --css /usr/local/apps/R/gcc_4.9.1/library/rmarkdown/rmarkdown/templates/html_vignette/resources/vignette.css --include-in-header /spin1/scratch/dalgleishjl/RtmpR3VvG1/rmarkdown-str62f25b19add6.html --mathjax --variable 'mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML'
output1 is a zipped copy of the results for test data and seems to have an html file that doesn't have complete output.

Output created: output_real.hichipper.qcreport.html
Warning messages:
1: Transformation introduced infinite values in continuous x-axis
2: Removed 13419 rows containing non-finite values (stat_bin).
Processing: d1
output1.zip

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.