quinlan-lab / pathoscore Goto Github PK

currently, in evaluate missing scores are essentially ignored. there should be a penalty for this.
I think one option is to scale each value in the roc output by the percent of scored variants, so if only 20% of variants are scored, then the maximum ROC curve will be 0.8

There is a new version of the vcf in clinvar, the file with the 20170802 data is archived. Here is the new line of code from the make.sh file with the new location

wget ftp://ftp.ncbi.nih.gov/pub/clinvar/vcf_GRCh37/vcf_2.0/archive/2017/clinvar_${date}.vcf.gz

from this paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5986715/

table S6 has > 7,000 scores from the experiment.

genomic positions for some of these are available in the other tables, probably easieast way is to generate a VCF with all genomic positions for PTEN, annotate with VEP, then create a lookup table from e.g. Ala328Gln to genomic position.

`

https://www.ncbi.nlm.nih.gov/pubmed/27749844

the scores don't seem to be available except via: miter.broadinstitute.org
might contact authors

we can implement this as filtering any variant that appears in the --exclude. currently this is done only for pathogenics to remove spurious pathogenics.

https://github.com/quinlan-lab/regionanalysis/blob/master/parvarfilter.py

Is the filter script. Frequency and genes can also be filtered with https://github.com/quinlan-lab/regionanalysis/blob/master/secondfilter.py

Can use it like:

python parvarfilter.py -x $DATA/clinvar-gnomad.txt -n clinvar -c -s patho -e gnomad -d genescreens/ad_genecards_clean.txt -f

Creates a file called $DATA/clinvar-patho-gnomad.txt ( you have to add back a vcf header, but that's an easy fix ).

python parvarfilter.py -x $DATA/gnomad-exac.txt -n gnomad -s benign -e exac -d genescreens/ad_genecards_clean.txt -f

Creates a set of gnomad benigns called gnomad-benign-exac.txt (gnomad, benign set, filtered on exac). Filters on AA change/allele matching. Also, optionally on AD gene set.

as in:
https://github.com/quinlan-lab/regionanalysis/blob/master/pathocompare.sh

accuracy * number of variants scored for each gene

see: http://home.chpc.utah.edu/~u6000771/pathoscore/testing/samocha.overview.html
where pLI has a higher bar for count of variants scored.

invocation: pathoscore.py evaluate -s cadd -s mpc -i mpc_regions -i blosum -s ccr_10_5 -s grantham -s pLI -s mis_badness -s combined spathogenic.vcf sbenign.vcf --prefix /uufs/chpc.utah.edu/common/home/u6000771/public_html/pathoscore/testing/samocha --functional

in practice, the most interesting variants will be in the very top percentile.

pathoscore should report a confusion matrix for the top-N or top-p-percentile variants by score. This will match how a user might filter.

https://content.iospress.com/articles/translational-science-of-rare-diseases/trd027

pathoscore should output 1 row per method with columns:

• method
• J
• se(J)
• TPR @ J
• FPR @ J
• AUC
• TP
• FP
• TN
• FN

https://www.nature.com/nature/journal/v515/n7526/full/nature13908.html
Supp Table 2?

ROC-curve isn't the most informative for this type of stuff. we should have precision-recall.

in a PR, add a separate file for clinvar benign and clinvar pathogenic to the truth/ directory.

it should be dated same as the original clinvar file. No need to remove common pathogenics as that is handled by pathoscore

now called tab10 and tab20. (reported by Edgar)