ePillID Benchmark

This repository contains data and code for ePillID - a benchmark for developing and evaluating computer vision models for pill identification. The ePillID benchmark is designed as a low-shot fine-grained benchmark, reflecting real-world challenges for developing image-based pill identification systems.

The characteristics of the ePillID benchmark include:

Reference and consumer images: The reference images are taken with controlled lighting and backgrounds, and with professional equipment. The consumer images are taken with real-world settings including different lighting, backgrounds, and equipment. For most of the pills, one image per side (two images per pill type) is available from the NIH Pillbox dataset.

Low-shot and fine-grained setting: 13k images representing 8184 appearance classes (two sides for 4092 pill types). For most of the appearance classes, there exists only one reference image, making it a challenging low-shot recognition setting.

The best performing model at the time of publication is a multi-head metric learning approach. Following a CNN encoder, a bilinear transformation layer is used to represent fine-grained embeddings.

Please refer to our paper for more details:

@inproceedings{usuyama2020epillid,
  title={ePillID Dataset: A Low-Shot Fine-Grained Benchmark for Pill Identification},
  author={Usuyama, Naoto and Delgado, Natalia Larios and Hall, Amanda K and Lundin, Jessica},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops},
  year={2020}
}

Data

Download and unzip ePillID_data.zip

Docker

Docker image https://hub.docker.com/r/naotous/pytorch-image

sudo nvidia-docker pull naotous/pytorch-image:py36torch041-legacy

Train and evaluate models

sudo nvidia-docker run --ipc=host -it -v $HOME:$HOME naotous/pytorch-image:py36torch041-legacy /bin/bash

use -v mount option to access folders from Docker containers

source activate py36

python train_cv.py --appearance_network resnet18 --max_epochs 10 --data_root_dir /home/naotous/ePillID_data

NOTE: max_epochs 10 for quick testing

Train and evaluate models on Azure ML

python azureml_submit_runs.py ePillID_0430

Notes and acknowledgments

The ePillID dataset includes customized data from two public NIH NLM databases. Please refer to their websites for original data:

For bilinear pooling implementations, we forked fast-MPN-Cov GitHub repo.

Please consider citing their work as well when you find our work useful in your research.

Disclaimer

This dataset and software are released for research purposes only.

gaiya2050 / epillid-benchmark Goto Github PK