nf-cmgg / germline Goto Github PK

Description of feature

Add the regions contained in the scatter BED files to the filenames as suggested by @matthdsm

Description of feature

Detect what genome is used through the FASTA file (removes the --genome parameter importance in seqplorer mode)

Description of the bug

Currently the SDF folder isn't checked to see if it exists, this should be added (it's only detected by the first process that needs it)

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of the bug

When running the pipeline multiple times, the multiqc folder also gets overwritten basically deleting all statistics of the files from the previously created VCFs

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of feature

https://github.com/frictionlessdata/tableschema-java

Description of feature

https://www.nextflow.io/docs/latest/channel.html#watchpath

=> possibly automatic samplesheet creation or automatic detection of the files without creating a samplesheet?

Description of feature

This flag should make mosdepth do a non-fast mode and output all files. When this flag is enabled, the callable input files will not be used.

Description of the bug

Currently params.genotype isn't actually part of the code. Add this check!

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of feature

Improve scattering so that the coverage is also taken into account before the variant calling.

Description of feature

Add vcf2db in "seqplorer" mode

Description of feature

Convert the scattering and header merging to processes

Description of feature

xref nf-core/modules#3006 (comment)

Perhaps we could merge all vcf's into a multisample per run, then run vep and split again. That way, we only need to run vep once, or once per chunk.

Description of feature

Add a feature that automatically looks in the outdir if the individuals VCFs were already generated by a previous run. Make this a toggle feature with a parameter which defaults to false

Description of feature

As alternative for GATK GenotypeGVCFs
https://samtools.github.io/bcftools/bcftools.html#convert

Description of the bug

Haplotypecaller is set to process_medium after which the setting are overrided in modules.conf to increase time and set cpu's to 1

This causes the job to request too much memory.
I suggest setting to process_single which should be enough for single chunks (pending execution report)

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of feature

Allow multiple bam inputs per sample to account for replicates, sequencing repeats and general extra coverage

Description of the bug

When dumping channels, the channel is emitted and doesn't go further to the set parameter. This should be fixed!

Ideally also make a common consensus for dump tag names

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of feature

Title says it all

Seqr doesn't handle annotations from the merged cache

https://github.com/CenterForMedicalGeneticsGhent/nf-cmgg-germline/blob/e2df7f00e81004db8516adfa0e5cbe0ac17a1333/workflows/nf-cmgg-germline.nf#L26

Description of feature

• Add ROI bed an callable region inputs to samplesheet instead of single BED
• Create intersect with callable regions and ROI if the ROI is given
• Create callable regions with mosdepth if no callable regions were given
• Create a param that can be used to specify a common ROI BED that will be used for all samples that have no ROI BED in their samplesheet

Description of feature

We could attempt to speed up the annotation by parallellizing the analysis.
cfr point 12 here https://www.ensembl.org/info/docs/tools/vep/script/vep_other.html

Description of feature

Add a dockerfile containing all the requirements for the pipeline, so we can run it as a single "tool" on nomad/galaxy/whatever

Description of the bug

When running a multi family run, the pipeline stalls after genomicsdbimport until all families have been run through genomicsdbimport

Command used and terminal output

No response

Relevant files

No response

System information

dev version on the 4th of January 2023

Description of feature

Add family_id field to samplesheet so we can define families without having to create ped files

Description of feature

The merge_vcf_headers.py script is relatively slow and memory intensive

Description of feature

The scatter count should scale with the family size to account for the multiple variants found per individual

Description of feature

We are standardizing our cluster to 8c/64gb so base.conf should reflect that

Description of feature

http://www.htslib.org/doc/samtools-index.html

Description of the bug

If the output file already exists in the output directory, it doesn't save the newer file. This should be fixed

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of feature

Improve bed_scatter_groovy to use less memory

Description of the bug

see title

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of the bug

Several join mismatches have been detected:

• Join when individuals got scattered (possibly after haplotypecaller since all these ran fine)
• Join when families got scattered (possibly after genomicsdbimport since most of those processes ran fine)
• Join when families aren't scattered (possibly before genomicsdbimport since the pipeline didn't get further than genotypegvcfs)
• ... (hopefully nowhere else 😁)

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of feature

ROI support has for now been disabled due to the addition of goleft_indexsplit. This should be solved in a new creative way

Description of feature

Also add a parameter to skip the genotyping and convert the gvcf to vcf using bcftools merge

Description of feature

Implement automatic scatter count detection based on the BED file. This way the pipeline will run in the most efficient way

Description of the bug

goleft_indexsplit makes more regions than it's supposed to when the reference contains more contigs than the scatter count => possible fix is to return to bed files to determine the regions and merge all alt contigs into one bed file and make bed files per all other regions

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of the bug

Caused by:
  Process `CMGG_CMGGGERMLINE:CMGGGERMLINE:ADD_PED_HEADER:RTGTOOLS_PEDFILTER (Proband)` terminated with an error exit status (1)

Command executed:

  rtg pedfilter \
      --vcf \
      Proband.samples.tsv \
  | rtg bgzip  - > Proband.vcf.gz
  
  
  cat <<-END_VERSIONS > versions.yml
  "CMGG_CMGGGERMLINE:CMGGGERMLINE:ADD_PED_HEADER:RTGTOOLS_PEDFILTER":
      rtgtools: $(echo $(rtg version | head -n 1 | awk '{print $4}'))
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead
  WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_NXF_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_NXF_DEBUG instead
  Error: Conflicting PED definitions of sex for individual -9

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of feature

Add a module that fetches data through DRS whenever a row in the samplesheet only contains a sample name

Description of the bug

It would be better for merged CRAM files to automatically regenerate a new BED file with mosdepth instead of merging the original ones. (This is less channel logic and will have better results)

Command used and terminal output

No response

Relevant files

No response

System information

No response

Description of feature

So we avoid having to run all the reference input steps.

cfr https://github.com/CenterForMedicalGeneticsGhent/nf-cmgg-preprocessing/blob/main/nextflow.config#L28-L34

Description of feature

Add an option to supply a samplesheet with called GVCFs to directly start genotyping and skipping the variant calling this way

Description of feature

Some pieces of code can be confusing when looked at the first time, add some more comments to all the code.

Description of question

Would it be better to run variant QC on a per sample base instead of genotyped family vcfs? This would provide more granular results IMO

Description of feature

Add truth_bed option to samplesheet

Description of feature

https://github.com/brentp/somalier

Description of feature

Add validation subworkflow which runs RTGtools and/or hap.py to generate a validation report and can be enabled with a flag or something

Description of feature

https://bedtools.readthedocs.io/en/latest/content/tools/jaccard.html

Interval similarity score

Description of feature

Add the posibility to exclude family and PED from the samplesheet and define that the sample isn't part of a family this way

Description of feature

Filter out families with only one sample before running the somalier subworkflow

Description of the bug

The label of genotypegvcfs is currently set on process_high. This is way too high, lower this!

Command used and terminal output

No response

Relevant files

No response

System information

No response