imed-lab / octa-net-octa-vessel-segmentation-network

OCTA-Net: A Two-stage Framework for Vessel Segmentation in OCTA Images

Paper | BibTex

ROSE: A Retinal OCT-Angiography Vessel Segmentation Dataset and New Model

Yuhui Ma, Huaying Hao, Jianyang Xie, Huazhu Fu, Jiong Zhang, Jianlong Yang, Zhen Wang, Jiang Liu, Yalin Zheng and Yitian Zhao

License

All rights reserved.

The code and data are released for academic research use only. For commercial use, please contact [email protected].

Condensed Abstract

Optical Coherence Tomography Angiography (OCTA) is a non-invasive imaging technique that has been increasingly used to image the retinal vasculature at capillary level resolution. However, automated segmentation of retinal vessels in OCTA has been under-studied due to various challenges such as low capillary visibility and high vessel complexity, despite its significance in understanding many vision-related diseases. In addition, there is no publicly available OCTA dataset with manually graded vessels for training and validation of segmentation algorithms. To address these issues, for the first time in the field of retinal image analysis we construct a dedicated Retinal OCTA SEgmentation dataset (ROSE), which consists of 229 OCTA images with vessel annotations at either centerline-level or pixel level. This dataset with the source code has been released for public access to assist researchers in the community in undertaking research in related topics. Secondly, we introduce a novel split-based coarse-to-fine vessel segmentation network for OCTA images (OCTA-Net), with the ability to detect thick and thin vessels separately. In the OCTA-Net, a split-based coarse segmentation module is first utilized to produce a preliminary confidence map of vessels, and a split-based refined segmentation module is then used to optimize the shape/contour of the retinal microvasculature. We perform a thorough evaluation of the state-of-the-art vessel segmentation models and our OCTA-Net on the constructed ROSE dataset. The experimental results demonstrate that our OCTA-Net yields better vessel segmentation performance in OCTA than both traditional and other deep learning methods. In addition, we provide a fractal dimension analysis on the segmented microvasculature, and the statistical analysis demonstrates significant differences between the healthy control and Alzheimer's Disease group. This consolidates that the analysis of retinal microvasculature may offer a new scheme to study various neurodegenerative diseases.

Installation

Clone this repo

git clone https://github.com/iMED-Lab/OCTA-Net-OCTA-Vessel-Segmentation-Network.git

Prerequisites

• python3
• numpy
• pillow
• opencv-python
• scikit-learn
• tensorboardX
• visdom
• pytorch
• torchvision

ROSE: A Retinal OCT-Angiography Vessel SEgmentation Dataset

The information about the ROSE dataset could be seen in the following link:

https://imed.nimte.ac.cn/dataofrose.html

Run the code

    cd code/OCTA-Net

Start Visdom

    python -m visdom.server

Coarse Stage:

    python front_main.py

Fine Stage:

    python fusion_main.py

Calculate Time Cost:

    python test_time.py

Calculate Metrics:

    cd ../../evaluation
    python evaluation.py

Calculate Time Cost:

    python draw_all_roc.py

Note:

train: --mode train

test: --mode test

SCC (ROSE-1): --dataset rose --data_dir ../../data/ROSE-1/SVC

DCC (ROSE-1): --dataset rose --data_dir ../../data/ROSE-1/DVC

WRCC (ROSE-1): --dataset rose --data_dir ../../data/ROSE-1/SVC_DVC

ROSE-2: --dataset cria --data_dir ../../data/ROSE-2

Citation

If you use this code for your research, please cite our papers.

@article{ma2021rose:,
  title={ROSE: a retinal OCT-angiography vessel segmentation dataset and new model},
  author={Ma, Yuhui and Hao, Huaying and Xie, Jianyang and Fu, Huazhu and Zhang, Jiong and Yang, Jianlong and Wang, Zhen and Liu, Jiang and Zheng, Yalin and Zhao, Yitian},
  journal={IEEE Transactions on Medical Imaging},
  year={2021},
  volume={40},
  number={3},
  pages={928--939},
  doi={10.1109/TMI.2020.3042802},
  publisher={IEEE}
}

octa-net-octa-vessel-segmentation-network's Issues

model training

Hi,
I want to know whether you are training two models in a single stage or the two models are trained separately?
Thanks!

about metric

Hi, first of all, thank you for your excellent work. But I have a few questions, first of all, about the performance evaluation, in the evaluation.py file, why do you have to do （dilated_gt_arr = cv2.dilate(gt_arr, kernel, iterations=1) first?

def numeric_score(pred_arr, gt_arr, kernel_size=(1, 1)):  # DCC & ROSE-2: kernel_size=(3, 3)
    """Computation of statistical numerical scores:

    * FP = False Positives
    * FN = False Negatives
    * TP = True Positives
    * TN = True Negatives

    return: tuple (FP, FN, TP, TN)
    """
    kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, kernel_size)
    dilated_gt_arr = cv2.dilate(gt_arr, kernel, iterations=1)
    
    FP = np.float(np.sum(np.logical_and(pred_arr == 1, dilated_gt_arr == 0)))
    FN = np.float(np.sum(np.logical_and(pred_arr == 0, gt_arr == 1)))
    TP = np.float(np.sum(np.logical_and(pred_arr == 1, dilated_gt_arr == 1)))
    TN = np.float(np.sum(np.logical_and(pred_arr == 0, gt_arr == 0)))
    
    return FP, FN, TP, TN


The second question is why do you do the ostu thresholding after getting the prediction? Does it make a difference if I set a threshold of 0.5?

‘’‘
    thresh_value, thresh_pred_img = cv2.threshold(pred_img, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
    print("shape of prediction", thresh_pred_img.shape)
    print("shape of groundtruth", gt_img.shape)
’‘’

Where to get "front_model-189-0.9019971996250081.pth" when using the fusion_main.py

as titling.

where is imed_models.py

Thank you for your excellent work, but I can't find imed_models.py in your code, where can I find it please?

The address of the pre-trained model is invalid

为什么没有验证集，或者说验证集和测试集是一样的？

when i run front_main.py i got the error: thick_gt = sample[2].to(device) IndexError: list index out of range

thick_gt = sample[2].to(device)

IndexError: list index out of range

This issue is likely due to a problem with my dataset？

IndexError: list index out of range

dataset path

Thank you very much for receiving your dataset. I encountered a problem when I replaced the data set path. Your default path is "/media/imed/data/mayuhui/TMI_OCTA_vseg/data/OCTA/CRIA", and I replaced it with "../../data/ROSE-2".and get infomation “IndexError: list index out of range”in code\OCTA-Net\train.py, line 17, in train_first_stage thin_gt = sample[2].to(device)。

That is, sampmle out of range.Hope you can answer, thank you very much！

About SCV_DVC in the first phase

Thank you very much for your excellent work. I would like to consult you for some questions. SCV_DVC files don't have thick_gt and thin_gt, so how can this dataset run the first stage network?

When I ran fusion_main.py, I got an error "expected Tensor as element 1 in argument 0, but got tuple"

@YuhuiMa Thank you for your wonderful work. When I ran fusion_main.py, I got an error "expected Tensor as element 1 in argument 0, but got tuple". Could you please tell me how to solve it? I am looking forward to your response.

dataset

Hi,
I tried asking for downloading dataset by sending email to [email protected] as stated in project website, but, still have not received answers yet. I also followed form as shown in the website. Could you tell me how I can access to the dataset?

fusion_model-370-0.9035.pth

Hi, I am trying to run tests but I get an error since it cannot find fusion_model-370-0.9035.pth. Where can I get this file?

keyError: 'auc'

when i run the coarse stage for ROSE-2, I get this error:

a similar issue was posted here but was closed without offering a solution. any idea how i can solve this issue?

Could you please share the pre-trained weights(gdrive links)

../resnest50-528c19ca.pth

在resnest.py文件中， model_path='../resnest50-528c19ca.pth' # /home/imed/OCT-A_segmentation/中的../resnest50-528c19ca.pth文件如何获取到？谢谢

Cria/Rose-2: my results are way worse than the paper's

I ran your code's first- and second stage and I am confused why I get such worse results than are described in the paper.

I ran the first stage with the resnest50-528c19ca.pth and I linked for my second stage with the [...]best-fusion.pth, [...]best-thick.pth and the [...]best-thin.pth models, generated by the first stage, to the suffix-variables.

Quantitative results for the fusion step:

AUC and ACC are fairly close, but DICE, the metric I find the most relevant, caps at around 0.41, which is way worse than what you wrote in your paper (0.7077)

Qualitative results for the fusion step:

We were interested in this paper because of the prospect of filtering out artifacts on OCTA images. These images above are probably the most extreme examples on the test set, but I don't see the models that I have trained, as acceptable enough to use.

Conclusion:
I have no trouble believing that I messed up somewhere, which would explain why I get such bad results. I just don't know where.

I have not used the front_model-189-0.9019971996250081.pth for the second stage, that you linked here. The download doesn't give me the file that I want but an .exe called BaiduNetdisk_7.19.0.18. I don't know what to do with that and I have no administrative rights here, so I can't launch the installation that it is trying to launch.
Could you maybe upload the front_model-189-0.9019971996250081.pth - like you did for resnest50-528c19ca.pth (#2) - on a drive? I would like to know if I can reproduce the results of your paper with that model.

Also concerning the front_model-189-0.9019971996250081.pth: Is this a fusion-, thin- or thick-model?

More details about environment

Program running problems程序运行问题

我运行了front_main.py，报错

Traceback (most recent call last):
File "e:/OCTASNet/code/OCTANet/front_main.py", line 64, in
first_net = train_first_stage(viz, writer, train_dataloader, first_net, first_optim, args.init_lr, thin_criterion, thick_criterion, device, args.power, epoch, args.first_epochs)
File "e:\OCTASNet\code\OCTANet\train.py", line 15, in train_first_stage
thin_gt = sample[2].to(device)
IndexError: list index out of range

为什么sample[2].to(device)就超范围了？

am i training it right?

I have been training this model and almost every time I get something similar to these results for the first stage and second stage
First stage (fusion, no thresh):

Second stage (fusion, no thresh):

The ground truth seems to be better represented by the first stage than by the second stage, which does not seem correct to me. Currently, to train the model I have tried two approaches:

1. change args.data_dir, run front_main.py and fusion_main.py for one of SVC, DVC, or SVC+DVC
2. change args.data_dir and run front_main.py for each SVC, DVC, and SVC+DVC. Afterwards, run fusion_main.py
   I would like to know if I am doing anything incorrectly or if there are some additional details I should keep in mind when training the model. Thank you

KeyError : auc

When i run the code,it shows KeyError: 'auc',i don't know how to solve it ?