Coder Social home page Coder Social logo

yangchao0053 / sam-med3d Goto Github PK

View Code? Open in Web Editor NEW

This project forked from uni-medical/sam-med3d

0.0 0.0 0.0 6.97 MB

SAM-Med3D: An Efficient 3D Model for Promptable Volumetric Medical Image Segmentation

License: Apache License 2.0

Shell 0.22% Python 99.78%

sam-med3d's Introduction

SAM-Med3D [Paper]

๐Ÿ’ช Recommendation

If you are interested in computer vision, we recommend checking out OpenGVLab for more exciting projects like SAM-Med2D!

๐ŸŒŸ Highlights

  • ๐Ÿ“š Curated the most extensive volumetric medical dataset to date for training, boasting 131K 3D masks and 247 categories.
  • ๐Ÿšค Achieved efficient promptable segmentation, requiring 10 to 100 times fewer prompt points for satisfactory 3D outcomes.
  • ๐Ÿ† Conducted a thorough assessment of SAM-Med3D across 15 frequently used volumetric medical image segmentation datasets.

๐Ÿ”จ Usage

Training

To train the SAM-Med3D model on your own data, follow these steps:

  1. Prepare Your Training Data: Ensure that your training data is organized according to the structure shown in the data/validation directories. Here is an example of how your file structure should look:
data/train
      โ”œโ”€โ”€ adrenal
      โ”‚ โ”œโ”€โ”€ ct_WORD
      โ”‚ โ”‚ โ”œโ”€โ”€ imagesTr
      โ”‚ โ”‚ โ”‚ โ”œโ”€โ”€ word_0025.nii.gz
      โ”‚ โ”‚ โ”‚ โ”œโ”€โ”€ ...
      โ”‚ โ”‚ โ”œโ”€โ”€ labelsTr
      โ”‚ โ”‚ โ”‚ โ”œโ”€โ”€ word_0025.nii.gz
      โ”‚ โ”‚ โ”‚ โ”œโ”€โ”€ ...
      โ”œโ”€โ”€ ...

Then modify the utils/data_paths.py according to your own data.

  1. Run the Training Script: Execute the following command in your terminal:
bash train.sh

This will start the training process of the SAM-Med3D model on your prepared data.

Evaluation

Prepare your own dataset and refer to the samples in data/validation to replace them according to your specific scenario. Then you can simply run bash infer.sh to test SAM-Med3D on your data. Make sure the masks are processed into the one-hot format (have only two values: the main image (foreground) and the background).

python validation.py --seed 2023\
 -vp ./results/vis_sam_med3d \
 -cp ./ckpt/sam_med3d.pth \
 -tdp ./data/validation -nc 1 \
 --save_name ./results/sam_med3d.py
  • vp: visualization path, dir to save the final visualization files
  • cp: checkpoint path
  • tdp: test data path, where your data is placed
  • nc: number of clicks of prompt points
  • save_name: filename to save evaluation results

For validation of SAM and SAM-Med2D on 3D volumetric data, you can refer to infer_sam.sh and infer_med2d.sh for details.

Hint: We also provide a simple script sum_result.py to help summarize the results from file like ./results/sam_med3d.py.

๐Ÿ”— Checkpoint

Our checkpoint of SAM-Med3D can be downloaded from Google Drive and Baidu NetDisk (pwd:r5o3).

Other checkpoints are available with their official link: SAM and SAM-Med2D.

๐Ÿ—ผ Method

๐Ÿ† Results

๐Ÿ’ก Overall Performance

Model Prompt Resolution Inference Time (s) Overall Dice
SAM N points 1024ร—1024ร—N 13 17.01
SAM-Med2D N points 256ร—256ร—N 4 42.75
SAM-Med3D 1 point 128ร—128ร—128 2 49.91
SAM-Med3D 10 points 128ร—128ร—128 6 60.94

Note: Quantitative comparison of different methods on our evaluation dataset. Here, N denotes the count of slices containing the target object (10 โ‰ค N โ‰ค 200). Inference time is calculated with N=100, excluding the time for image processing and simulated prompt generation.

๐Ÿ’ก Dice on Different Anatomical Architecture and Lesions

Model Prompt A&T Bone Brain Cardiac Gland Muscle Seen Lesion Unseen Lesion
SAM N points 17.19 22.32 17.68 2.82 11.62 3.50 12.03 8.88
SAM-Med2D N points 46.79 47.52 19.24 32.23 43.55 35.57 26.08 44.87
SAM-Med3D 1 point 46.80 54.77 34.48 46.51 57.28 53.28 42.02 40.53
SAM-Med3D 10 points 55.81 69.13 40.71 52.86 65.01 67.28 50.52 48.44

Note: Comparison from the perspective of anatomical structure and lesion. A&T represents Abdominal and Thorax targets. N denotes the count of slices containing the target object (10 โ‰ค N โ‰ค 200).

๐Ÿ’ก Visualization

๐Ÿ—“๏ธ Ongoing

  • Dataset release
  • Train code release
  • [Feature] Evaluation on 3D data with 2D models (slice-by-slice)
  • Evaluation code release
  • Pre-trained model release
  • Paper release

๐ŸŽซ License

This project is released under the Apache 2.0 license.

๐Ÿ’ฌ Discussion Group

image

BTW, welcome to follow our Zhihu official account, we will share more information on medical imaging there.

๐Ÿ™ Acknowledgement

  • We thank all medical workers and dataset owners for making public datasets available to the community.
  • Thanks to the open-source of the following projects:

๐Ÿ‘‹ Hiring & Global Collaboration

  • Hiring: We are hiring researchers, engineers, and interns in General Vision Group, Shanghai AI Lab. If you are interested in Medical Foundation Models and General Medical AI, including designing benchmark datasets, general models, evaluation systems, and efficient tools, please contact us.
  • Global Collaboration: We're on a mission to redefine medical research, aiming for a more universally adaptable model. Our passionate team is delving into foundational healthcare models, promoting the development of the medical community. Collaborate with us to increase competitiveness, reduce risk, and expand markets.
  • Contact: Junjun He([email protected]), Jin Ye([email protected]), and Tianbin Li ([email protected]).

sam-med3d's People

Contributors

blueyo0 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.