Comments (3)
Thanks Thuy - I'd be curious how quickly the smallest available cram would run through the pipeline. We may still want to downsample to get the runtime down to 30-60 minutes for quicker development iteration.
from pipelines.
@kbergin I've tried to combine #25 and #26 with the same pull request: #28. We've provided the smallest example lanelet crams from an actual vector data release (AG1K Phase3). I'm a bit worried that they don't actually represent a typical vector sample since they are 3 orders of magnitude smaller. However, we can use them as examples of what a header and format of a Core Sequencing cram will look like.
I'll need to consult with @alimanfoo and @hardingnj on which samples with lanelet crams would provide a better representation of the typical vector sample so that we can compare pipeline test outputs with confidence.
from pipelines.
From slack conversation with @kbergin Jun 3, 2020:
So we usually look for:
- Plumbing tests, automated on small data that runs quickly but still works thorugh the whole pipeline
- Scientific tests - these would be on a more normal size sample, maybe a few, that can take longer to run but ideally a day or less and could be automated on PRs to master before release, or could be manually kicked off when needed
- We can also do an even larger set of data to fully validate the pipeline for this first pass, but this could be the same as the scientific test data depending on your preference.
- For testing upgrades to infrastructure we usually run much larger scale tests on many samples but that can be a later thing.
from pipelines.
Related Issues (20)
- Define read group contents in vector short read alignment pipelines HOT 1
- Input source for mosquito short read alignment pipeline HOT 3
- Validation/truth datasets for variant calling calibration, comparison and evaluation HOT 26
- Copy script to convert vcf to zarr to this repo
- Port baseline Plasmodium amplicon SNP calling pipeline. HOT 1
- Add platform argument to build_pipeline_release script HOT 2
- How to make bwa short read alignment deterministic for testing HOT 1
- Mosquito phasing pipeline development HOT 22
- Genotype Concordance Validation of genotyping pipelines HOT 10
- Create pipelines that download multiple samples from IRODS in a batch
- Determine final lanelet manifest fields required for vector alignment and genotyping HOT 1
- Cromwell configurations on farm5 are using localization = copy, which explodes size of workflow working directory
- Convert cram to bam in vector alignment pipeline
- malaria-dk lsf group deprecated
- Output of vector short read alignment pipeline is named *.bam.bam HOT 1
- Use flexible fields in manifest
- Validate results for phasing pipeline HOT 17
- Ag3.0 phasing pipeline production run HOT 15
- Number of SNPs and Indels doesn't match with the numbers reported in README files HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pipelines.