irischang020 / deepprojection

An X-ray single particle imaging classifier.

Deepprojection

Deepprojection is a X-ray scattering pattern classifier for the detection of "single hit" in single particle imaging (SPI) experiments.

Installation

• Direct from github with pip

  pip install git+https://github.com/carbonscott/deepprojection --upgrade --user

• Download and pip

  cd <path_to_deepprojection>    # e.g. cd $HOME/codes/deepprojection
  
  pip install . --user

• Download and change PYTHONPATH

  export PYTHONPATH="<path_to_deepprojection>:$PYTHONPATH"

Install dependencies with conda

Create an environment at a custom location for the project

conda create --prefix <path> python=3.7

Install psana

conda install -c lcls-rhel7 psana-conda

Install other packages

• pytorch

  conda install pytorch=1.4.0=py3.7_cuda10.1.243_cudnn7.6.3_0 -c pytorch

• tqdm

  conda install tqdm=4.56.2=pyhd8ed1ab_0 -c conda-forge

• skimage

  conda install scikit-image=0.18.1=py37hdc94413_0 -c conda-forge

Datasets

Three scenarios

Each scattering image, captured by a PnCCD detector, consists of four panels. Deepprojection classifier is able to perform training and inference on panels, stacks of panels, and whole images, respectively.

Experiment vs Simulation

The classifier has been tested on both experimental and simulated images. In particular, simulation is performed by skopi.

Run examples

All three scenarios in the above section are supported. Panels of simulated images are used in the following demo.

• Specify metadata, e.g. basename, path and labels associated with SPI image files, in a .csv file.

  basename,label,path
  5A1A.1_hit,1,/reg/data/ana03/scratch/cwang31/scratch/skopi/h5s
  6KPI.3_hit,2,/reg/data/ana03/scratch/cwang31/scratch/skopi/h5s
  6M54.1_hit,1,/reg/data/ana03/scratch/cwang31/scratch/skopi/h5s
  6TSH.3_hit,2,/reg/data/ana03/scratch/cwang31/scratch/skopi/h5s
  6XLU.1_hit,1,/reg/data/ana03/scratch/cwang31/scratch/skopi/h5s
  6ZK9.4_hit,2,/reg/data/ana03/scratch/cwang31/scratch/skopi/h5s
  7DDE.1_hit,1,/reg/data/ana03/scratch/cwang31/scratch/skopi/h5s
  7FIE.2_hit,2,/reg/data/ana03/scratch/cwang31/scratch/skopi/h5s
  7K3W.3_hit,2,/reg/data/ana03/scratch/cwang31/scratch/skopi/h5s

• Train.

  python train.simulated_panel.py

  A log file with a time stamp will be generated under logs, such as <timestamp>.train.log

• Check learning curves during training. Open alog.learncurve.py and change the time stamp in the example file.

  python alog.learncurve.py

• Validate. Open validate.simulated_panel.query.py and change the time stamp in the example file.

  python validate.simulated_panel.query.py
  

  Query is one way of validating model performance, which is threshold free.

• Print confusion matrix. Open alog.validate.query.py and change the time stamp in the example file.

  python alog.validate.query.py

  Sample output (Two categories in simulated datasets, four in experimental datasets):

                 single hit   multi hit    accuracy   precision      recall specificity          f1
  -------------------------------------------------------------------------------------------------
  single hit  |        1845         174        0.92        0.91        0.92        0.91        0.92
   multi hit  |         159        1822        0.92        0.92        0.91        0.92        0.92