jakelever / civicmine

Text mining cancer biomarkers for the CIVIC database

CIViCmine

This is a text mining project to assist curation in the Clinical Interpretation of Variants in Cancer (CIViC) database. CIViC catalogs information on diagnostic, predictive, predisposing and prognostic variants in cancer. This project aims to text mine this data from PubMed and Pubmed Central Open Access subset

In brief: it finds sentences that describe genes, variants, cancer types and optionally drugs. We then annotated many of them to create training data. This is then used to build a classifier using the Kindred package. This is then applied to all of PubMed and PubMed Central Open Access to extract sentences and structured information. This is then collated for download and viewing through a web viewer.

System Requirements

This is a Python3 project which has been tested on Centos 6/7 but should work on other Linux operating systems and MacOS. An individual process of this can be run on a laptop or desktop computer. But in order to process all of the literature (PubMed, etc), this should really be run on a cluster or server-like machine. A cluster that uses Slurm or the SunGrid engine (SGE) are supported. Each node needs only 4 GBs on RAM.

This project relies on text mining using Kindred and Snakemake. These can be installed through pip.

It uses biomedical text converted using BioText.

Installation Guide

You can clone this repo using Git or download the ZIP file of it.

git clone https://github.com/jakelever/civicmine.git

It uses the BioText project which downloads and converts the biomedical research literature into the BioC XML format. You'll need to clone it and run the conversion scripts.

The code dependencies can be installed with the command below. You also need to install the scispacy language model.

pip install kindred snakemake scispacy zenodo_get
pip install https://s3-us-west-2.amazonaws.com/ai2-s2-scispacy/releases/v0.4.0/en_core_sci_sm-0.4.0.tar.gz

Installation should take a maximum of 15 minutes (mostly due to the Spacy and language models installation).

Instructions for Use

A full run of CIViCmine will take some time as the large corpora need to be downloaded and parsing takes a long time. A cluster is strongly recommended for this task.

A test run (which is what the Travis-CI test does) can be executed with:

MODE=test snakemake --cores 1

The full run can be executed after you've downloaded and converted the BioText data with the command below, pointing BIOTEXT to the directory with the BioText BioC XML files.

MODE=full BIOTEXT=$BIOTEXT snakemake --cores 1

Practically, you'll likely want to use a cluster to parallelize this. Please refer to the Snakemake documentation for information about how to use a cluster.

For uploading the output to Zenodo, this project uses bigzenodo and the submission.json file.

Inputs

The project uses wordlists from the BioWordlists project for cancers, genes, drugs, variants and conflicting terms. The corpora used are PubMed abstracts and full-text papers from the PubMed Central Open Access Subset. While processing the full-text articles, subsection headers (e.g. Results) are also extracted to make it easier to locate where statements are made. The list of possible headers extracted can be found here.

We have annotated 800 sentences that comprise the civicmine_corpus which are used for training a Kindred classifier. This data can be found in the data/ directory and has been split into training and test sets.

Outputs

The three output files of CIViCmine is outlined below. You likely want civicmine_collated.tsv if you just want the list of cancer biomarkers. If you want the supporting sentences, look at civicmine_sentences.tsv. You can use the matching_id column to connect the two files. If you want to dig further and are okay with a higher false positive rate, look at civicmine_unfiltered.tsv.

civicmine_collated.tsv: This contains the cancer biomarkers with citation counts supporting them. It contains the normalized cancer and gene names along with IDs for HUGO, Entrez Gene and the Disease Ontology.

civicmine_sentences.tsv: This contains the supporting sentences for the cancer biomarker in the collated file. Each row is a single supporting sentence for one cancer biomarker. This file contains information on the source publication (e.g. journal, publication date, etc), the actual sentence and the cancer biomarker extracted.

civicmine_unfiltered.tsv: This is the raw output of the applyModelsToSentences.py script across all of PubMed, Pubmed Central Open Access and PubMed Central Author Manuscript Collection. It contains every predicted relation with a prediction score above 0.5. So this may contain many false positives. Each row contain information on the publication (e.g. journal, publication date, etc) along with the sentence and the specific cancer biomarker extracted (with HUGO, Entrez Gene and Disease Ontology IDs). This file is further processed to create the other two.

Shiny App

The code in shiny/ is the Shiny code used for the web viewer. If it is helpful, please use the code for your own projects. The list of dependencies is found at the top of the app.R file.

Paper

The code to generate all the figures and text for the paper can be found in paper/. This may be useful for generating an up-to-date version of the plots for a newer version of CIViCmine.

Changelog

• v6 data release: change to Kindred's EntityRecognizer uses strict string matching instead of token matching, so results are minorly different
• v11 data release: updated the Biowordlists to expand the drug name list
• v22 data release: Reworked pipeline to use Biotext+snakemake instead of PubRunner. Prioritized using text from PMC. Also updated the Biowordlist

Citing

The paper is now published in Genome Medicine with the preprint still available at bioRxiv. It'd be wonderful if you would cite the paper if you use the methods or data set.

@article{lever2018civicmine,
  title={Text-mining clinically relevant cancer biomarkers for curation into the CIViC database},
  author={Lever, Jake and Jones, Martin R and Danos, Arpad M and Krysiak, Kilannin and Bonakdar, Melika and Grewal, Jasleen and Culibrk, Luka and Griffith, Obi L and Griffith, Malachi and Jones, Steven JM},
  journal={Genome Medicine},
  volume={11},
  number={78},
  year={2019},
  publisher={BioMed Central}
}

Issues

If you encounter any problems, please file an issue along with a detailed description.

civicmine's Issues

Annotated Sentences in CivicMine

HI @jakelever ,

I read the CivicMine paper and came across the following mentioned in the Abstract -

To this end, a group of cancer genomics experts annotated biomarkers and their clinical associations discussed in 800 sentences and achieved good inter-annotator agreement

May I know if the corpus linked by @swartchris8 in #2 (https://github.com/jakelever/cancermine/blob/master/data/cancermine_corpus.zip) is the one that contains these 800 sentences? Were these sentences were manually annotated by experts?

If not can you point me to the corpus of these 800 manually annotated sentences?

search gene by free text

Hi
Great repo!
I added an option to search by gene using a free text query.
If this is interesting I can make a PR.

Error when running "pubrunner --test ."

Hello Jake,
I have set up the environment for civicmine as described in your Github. I am trying to replicate the analysis with "pubrunner --test ." but I ran into some issues. Here is the output and error description:

---

Working directory: /home/kalifa/pubrunner/workspace/CIViCmine/test
Traceback (most recent call last):
File "/home/kalifa/.local/bin/pubrunner", line 33, in
sys.exit(load_entry_point('pubrunner==0.5.3', 'console_scripts', 'pubrunner')())
File "/home/kalifa/.local/lib/python3.7/site-packages/pubrunner/command_line.py", line 66, in main
pubrunner.pubrun(args.codebase,args.test,(not args.nogetresource),forceresource_dir=args.forceresource_dir,forceresource_format=args.forceresource_format,outputdir=args.outputdir)
File "/home/kalifa/.local/lib/python3.7/site-packages/pubrunner/pubrun.py", line 352, in pubrun
prepareConversionAndHashingRuns(toolSettings,mode,workingDirectory)
File "/home/kalifa/.local/lib/python3.7/site-packages/pubrunner/pubrun.py", line 86, in prepareConversionAndHashingRuns
eutilsToFile('pubmed',pmid,filename)
File "/home/kalifa/.local/lib/python3.7/site-packages/pubrunner/pubrun.py", line 41, in eutilsToFile
f.write(xml)
File "/usr/lib/python3.7/codecs.py", line 721, in write
return self.writer.write(data)
File "/usr/lib/python3.7/codecs.py", line 377, in write
data, consumed = self.encode(object, self.errors)
TypeError: utf_8_encode() argument 1 must be str, not bytes

I am using python 3.7 and Ubuntu 19.10. The problem is briefly solved if I change the following line of codes in the codecs.py file.
def write(self, object):

    """ Writes the object's contents encoded to self.stream.
    """
    data, consumed = self.encode(object.decode('uft-8'), self.errors)
    self.stream.write(data)

However, other problems with snakemake rise up, therefore I have to undone the changes. I have been working on this for the past two days. I will appreciate it if you can render me some support.
Thanks.

Corpus

Hi Jake,
Quick question: You have four categories (diagnostic, predictive, predisposing, prognostic). Do you by any chance have any plans to classify the corpus with whether a relationship in a sentence is a true positive or false positive? OR, do you know of a corpus like this out in the interwebs?
Thanks,
KMS

Can't find T790M mutation in civicmine

Hi jakelever,

Thanks for this wonderful project.

When i used the civicmine (http://bionlp.bcgsc.ca/civicmine) i can't find "T790M" in any sentence. It was odd for me because EGFR T790M is very famous biomarker in treatment cancer.

This is a tokenizer problem that Spacy language model (en_core_web_sm) tokenizes the "T790M" as a "T790" and "M". (('T790', 'NOUN'), ('M', 'PROPN'))

I changed the kindred package like this (kindred/Parser.py)

if not model in Parser._models:
      Parser._models[model] = spacy.load(model, disable=['ner'])

      self.nlp = Parser._models[model]
      special_case = [{ORTH: "T790M"}]
      self.nlp.tokenizer.add_special_case("T790M", special_case)

Now "T790M" is ('T790M', 'VERB') fixed.

best,
jakelever

Access to annotated sentences

I was wondering where I can find the 1500 annotated sentences used to train the model?

I would assume these are the sentences: https://github.com/jakelever/cancermine/blob/master/data/cancermine_corpus.zip