futianfan / clinical-trial-outcome-prediction

benchmark dataset and Deep learning method (Hierarchical Interaction Network, HINT) for clinical trial approval probability prediction, published in Cell Patterns 2022.

HINT: Hierarchical Interaction Network for Clinical Trial Outcome Prediction

This repository hosts HINT, a deep learning based method for clinical trial outcome prediction. The repository can be mainly divided into three parts:

• benchmark describes the process of curating benchmark dataset named Trial Outcome Prediction (TOP) for clinical trial outcome prediction.
• HINT is the Hierarchical Interaction Network, a deep learning based method.
• data stores processed data.

The following figure illustrates the pipeline of HINT.

Table Of Contents

• Installation
• Benchmark Data
• HINT: Learn and Inference
• Tutorial (Jupyter Notebook)
• Contact

---

⚙️ Installation

We build conda environment and uses conda or pip to install the required packages. See conda.yml for all the packages.

conda create -n predict_drug_clinical_trial python==3.7 
conda activate predict_drug_clinical_trial 
conda install -c rdkit rdkit  
pip install tqdm scikit-learn 
pip install torch
pip install seaborn 
pip install icd10-cm

We use following command to activate conda environment.

conda activate predict_drug_clinical_trial

---

📊 Benchmark Data

To standardize the clinical trial outcome prediction, we create a benchmark dataset for Trial Outcome Prediction named TOP, which incorporate rich data components about clinical trials, including drug, disease and protocol (eligibility criteria). All the scripts are in the folder benchmark. Please see benchmark/README.md for details.

---

🤖 HINT: Learn and Inference

After processing the data, we learn the Hierarchical Interaction Network (HINT) on the following four tasks. The following figure illustrates the pipeline of HINT. All the scripts are available in the folder HINT. Please see HINT/README.md for details.

Prediction results

We add the prediction results in ./results for all the three phases.

Trained model

The trained HINT models for all the three phases are available in ./save_model.

📚 Tutorial (jupyter notebook)

• benchmark: tutorial_benchmark.ipynb describes some key components of the data curation process.
• HINT: tutorial_HINT.ipynb is a tutorial to learn and evaluate HINT step by step.

📞 Contact

Please contact [email protected] for help or submit an issue. This is a joint work with Kexin Huang, Cao(Danica) Xiao, Lucas M. Glass and Jimeng Sun.

Benchmark Usage Agreement

The benchmark dataset and code (including data collection and preprocessing, model construction, learning process, evaluation), referred as the Works, are publicly available for Non-Commercial Use only at https://github.com/futianfan/clinical-trial-outcome-prediction. Non-Commercial Use is defined as for academic research or other non-profit educational use which is: (1) not-for-profit; (2) not conducted or funded (unless such funding confers no commercial rights to the funding entity) by an entity engaged in the commercial use, application or exploitation of works similar to the Works; and (3) not intended to produce works for commercial use.

clinical-trial-outcome-prediction's Issues

Missing package in conda.yml

When I run src/learn_phaseI.py

File "/home/user/reproduce/clinical-trial-outcome-prediction/src/utils.py", line 12, in plot_hist
import seaborn as sns
ModuleNotFoundError: No module named 'seaborn'

no module named from graph_visualize_interpret

import data2graph

Are the multiple drugs in the column `drug` in data files the group of drugs being compared in the trial at the same time?

Thanks in advance!

a few questions

Hi, Thanks for open sourcing this. I have a few questions regarding the paper:

1. How the outcome label is determined? It seems the outcome label comes from IQVIA so the process of generating it was not shown in this repo. In the paper, it was described as The trial is labelled as success if p-value is less than 0.05 and negative if p-value is higher than 0.05. I am not sure if this is also applicable for Phase I trials. Could you elaborate more on that?
2. How does ADMET pretraining help with Phase II prediction? Since majority of Phase II trials failed due to efficacy, I am not sure why pretraining ADMET could help with the predictions. Instead, understanding the disease-target relation may help.

drugdataloader==0.1

When I run conda env create -f conda.yml, it returns:

Pip subprocess error:
ERROR: Could not find a version that satisfies the requirement drugdataloader==0.1 (from -r /Users/zzachw/Projects/code_review/clinical-trial-outcome-prediction/condaenv.rzi2syk0.requirements.txt (line 10)) (from versions: none)
ERROR: No matching distribution found for drugdataloader==0.1 (from -r /Users/zzachw/Projects/code_review/clinical-trial-outcome-prediction/condaenv.rzi2syk0.requirements.txt (line 10))

Unicode error

Hi,

thank you for your precious work.

I am trying to reproduce tutorial_HINT.ipynb but I found the following error

`---------------------------------------------------------------------------
UnicodeDecodeError Traceback (most recent call last)
in
----> 1 train_loader = csv_three_feature_2_dataloader(train_file, shuffle=True, batch_size=32)
2 valid_loader = csv_three_feature_2_dataloader(valid_file, shuffle=False, batch_size=32)
3 test_loader = csv_three_feature_2_dataloader(test_file, shuffle=False, batch_size=32)

~\Desktop\CLINICALTRIAL\HINT\dataloader.py in csv_three_feature_2_dataloader(csvfile, shuffle, batch_size)
103 def csv_three_feature_2_dataloader(csvfile, shuffle, batch_size):
104 with open(csvfile, 'r') as csvfile:
--> 105 rows = list(csv.reader(csvfile, delimiter=','))[1:]
106 ## nctid,status,why_stop,label,phase,diseases,icdcodes,drugs,smiless,criteria
107 nctid_lst = [row[0] for row in rows]

~\Miniconda3\envs\predict_drug_clinical_trial\lib\encodings\cp1252.py in decode(self, input, final)
21 class IncrementalDecoder(codecs.IncrementalDecoder):
22 def decode(self, input, final=False):
---> 23 return codecs.charmap_decode(input,self.errors,decoding_table)[0]
24
25 class StreamWriter(Codec,codecs.StreamWriter):

UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1027: character maps to `

Do you know how could I solve it?

Regards,
Federico

Best Way to Replicate Results

Hey! I was wondering what's the best way to replicate the results of this project and predict clinical trial outcomes? Is there any specific file or code I should start with?

cannot create the env from conda.yml

when I am running the command: conda env create -f conda.yml, it returns

Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:
  - ninja==1.9.0=py37h04f5b5a_0
  - readline==8.0=h1de35cc_0
  - zlib==1.2.11=h1de35cc_3
  - gettext==0.19.8.1=hb0f4f8b_2
  - tk==8.6.10=hb0a8c7a_0
  - freetype==2.10.2=ha233b18_0
  - python==3.7.7=hf48f09d_4
  - libxslt==1.1.34=h83b36ba_0
  - lz4-c==1.9.2=h0a44026_0
  - llvm-openmp==10.0.0=h28b9765_0
  - cffi==1.14.0=py37hc512035_1
  - openssl==1.1.1h=haf1e3a3_0
  - py-boost==1.67.0=py37h6440ff4_4
  - ...

Obtain the Full drugbank data

Great Work! Just one question: how can we get the full information for the drugbank data?
I noticed that it can be downloaded from drugbank website, https://go.drugbank.com/releases/latest, but I am not sure how to pre-process it after we obtain the full database.xml file. Thanks!

Where can I possibly find the complete version of `drugbank_mini.csv`?

Hi, Thanks again for curating this dataset! I was just wondering where I can find the complete version of drugbank_mini.csv, since using drug names as query through drugbank's vocabulary file does not always output a valid drugbank id. drugbank_mini.csv contains more information and might be useful to map those ambiguous ones to correct ids.

no module named graph_visualize_interpret

from graph_visualize_interpret import data2graph

Obtain the Full drugbank data

Thanks for the great work! Unfortunately the Dropbox link in #7 seems no longer valid? If so, would you mind re-uploading it? Many thx!