Third place solution to the MedAI: Transparency in Medical Image Segmentation challenge.
The link to the paper is available at: https://journals.uio.no/NMI/article/view/9157
The challenge overview is available at: https://www.nora.ai/Competition/image-segmentation.html
For our experiments, we make use of the Medical Image Segmentation with Convolutional Neural Networks (MIScnn) open-source python library: https://github.com/frankkramer-lab/MIScnn. All experiments were programmed with Tensorflow using Keras backend.
The repository contains:
- Training scripts
- Model
- Postprocessing
Model weights, training curves and dataset partitions may be found at: https://drive.google.com/drive/folders/1M1JOeH1Cra4f8HPmfnv13uwE5Is6dlGi?usp=sharing
The Attention U-Net incorporates Attention Gates (AGs) into U-Net, which uses a gating signal that aggregates multiscale spatial information to highlight salient featurespresent in skip connection. We leverage the interpretable nature of the Attention U-Net, with the performance benefits of transfer learning, combining a ResNet152 encoder pre-trained on ImageNet with an Attention U-Net decoder network:
Images obtained from colonoscopy are used as input into models in various orientations (test-time augmentation). Five Attention U-Nets individually generate predictions, and softmax activations are then thresholded then averaged over all networks. Segmentation masks are converted to binary labels, resized to the original image resolution, and connected component analysis removes any regions occupying <1% of the image, generating the final prediction.
To investigate how the pipeline generates the final segmentation map, attention coefficients generated by the Attention Gates (AG) in each Attn U-Net may be visualised for:
- Polyp segmentation
- Instrument segmentation
An example of a well segmented case in the test set is shown for:
- Polyp segmentation
- Instrument segmentation
We compare pipeline predictions with images from the hold-out test set, investigating the cause, and proposing solutions, where the model fails to provide accurate polyp (a-c) and instrument (d-f) segmentations: