htylab / abdomenatlas

[NeurIPS 2023] AbdomenAtlas

AbdonmenAtlas-8K

We are proud to introduce AbdomenAtlas-8K, a substantial multi-organ dataset with the spleen, liver, kidneys, stomach, gallbladder, pancreas, aorta, and IVC annotated in 8,448 CT volumes, totaling 3.2 million CT slices.

An endeavor of such magnitude would demand a staggering 1,600 weeks or roughly 30.8 years of an experienced annotator's time.

In contrast, our annotation method has accomplished this task in three weeks (premised on an 8-hour workday, five days a week) while maintaining a similar or even better annotation quality.

Paper

AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weeks
Chongyu Qu¹, Tiezheng Zhang¹, Hualin Qiao², Jie Liu³, Yucheng Tang⁴, Alan L. Yuille¹, and Zongwei Zhou^1,*
1 Johns Hopkins University,
² Rutgers University,
³ City University of Hong Kong,
⁴ NVIDIA
NeurIPS 2023
paper | code | dataset | annotation

AbdomenAtlas-8K: Human-in-the-Loop Annotating Eight Anatomical Structures for 8,448 Three-Dimensional Computed Tomography Volumes in Three Weeks
Chongyu Qu¹, Tiezheng Zhang¹, Hualin Qiao², Jie Liu³, Yucheng Tang⁴, Alan L. Yuille¹, and Zongwei Zhou^1,*
1 Johns Hopkins University,
² Rutgers University,
³ City University of Hong Kong,
⁴ NVIDIA
RSNA 2023 (Oral Presentation)
paper | code | slides

★ We have maintained a document for Frequently Asked Questions.

0. Installation

git clone https://github.com/MrGiovanni/AbdomenAtlas

See installation instructions to create an environment and obtain requirements.

1. Download AI models

We offer pre-trained checkpoints of Swin UNETR and U-Net. The models were trained on a combination of 14 publicly available CT datasets, consisting of 3,410 (see details in CLIP-Driven Universal Model). Download the trained models and save them into ./pretrained_checkpoints/.

Architecture	Param	Download
U-Net	19.08M	link
Swin UNETR	62.19M	link

2. Prepare your datasets

It can be publicly available datasets (e.g., BTCV) or your private datasets. Currently, we only take data formatted in nii.gz. This repository will help you assign annotations to these datasets, including 25 organs and six types of tumors (where the annotation of eight organs is pretty accurate).

2.1 Download

Taking the BTCV dataset as an example, download this dataset and save it to the datapath directory.

cd $datapath
wget https://www.dropbox.com/s/jnv74utwh99ikus/01_Multi-Atlas_Labeling.tar.gz
tar -xzvf 01_Multi-Atlas_Labeling.tar.gz

2.2 Preprocessing

Generate a list for this dataset.

cd AbdomenAtlas/
python -W ignore generate_datalist.py --data_path $datapath --dataset_name $dataname --folder img --out ./dataset/dataset_list --save_file $dataname.txt

3. Generate masks

U-Net

CUDA_VISIBLE_DEVICES=0 python -W ignore test.py --resume pretrained_checkpoints/unet.pth --backbone unet --save_dir $savepath --dataset_list $dataname --data_root_path $datapath --store_result >> logs/$dataname.unet.txt

Swin UNETR

CUDA_VISIBLE_DEVICES=0 python -W ignore test.py --resume pretrained_checkpoints/swinunetr.pth --backbone swinunetr --save_dir $savepath --dataset_list $dataname --data_root_path $datapath --store_result >> logs/$dataname.swinunetr.txt

To generate attention maps for the active learning process (Step 5 [optional]), remember to save entropy and soft predictions by using the options --store_entropy and --store_soft_pred

4. Data Assembly

In the assembly process, our utmost priority is given to the original annotations supplied by each public dataset. Subsequently, we assign secondary priority to the revised labels from our annotators. The pseudo labels, generated by AI models, are accorded the lowest priority. The following code can implement this priority into the assembled dataset.

python -W ignore assemble.py --data_path $savepath --dataset_name $dataname --backbone swinunetr --save_dir SAVE_DIR --version V1

This is how our AbdonmenAtlas-8K appears

    $savepath/
    ├── $dataname_img0001
    ├── $dataname_img0002
    ├── $dataname_img0003
        │── ct.nii.gz
        ├── original_label.nii.gz
        ├── pseudo_label.nii.gz
        └── segmentations
            ├── spleen.nii.gz
            ├── liver.nii.gz
            ├── pancreas.nii.gz

5. [Optional] Active Learning

If you want to perform the active learning process, you will need the following active learning instructions to generate the attention map for human annotators.

Figure. Illustration of an attention map.

TODO

• Release pre-trained AI model checkpoints (U-Net and Swin UNETR)
• Release the AbdomenAtlas-8K dataset (we commit to releasing 3,410 of the 8,448 CT volumes)
• Support more data formats (e.g., dicom)

Citation

@article{qu2023annotating,
  title={Annotating 8,000 Abdominal CT Volumes for Multi-Organ Segmentation in Three Weeks},
  author={Qu, Chongyu and Zhang, Tiezheng and Qiao, Hualin and Liu, Jie and Tang, Yucheng and Yuille, Alan and Zhou, Zongwei},
  journal={arXiv preprint arXiv:2305.09666},
  year={2023}
}

Acknowledgements

This work was supported by the Lustgarten Foundation for Pancreatic Cancer Research and partially by the Patrick J. McGovern Foundation Award. We appreciate the effort of the MONAI Team to provide open-source code for the community.