The aim of the project was to achieved binary segmentation of brain tumoral MRI (magnetic resonance imaging) images. For that purpose two models were used: pretrained DeepLabV3 and AGResUNet. Both use a encoder-decoder arquitecture and are fully convolutional networks. The quality of the models were measured using dice, precision, recall, accuracy and IoU, typically use in this kind of problems.
The data set used consisted in a set of two-dimensional images of brain regions extracted through magnetic resonance imaging. We had several types of images or sequences based on the parameters configured during the acquisition of these.
- T1-weighted (T1): distinguishes between white and gray matter. Helps detect the presence of hemorrhages.
- T2-weighted (T2): allows detection of regions with fluid, aiding in the detection of edemas and inflamed tissues.
- T1-weighted with contrast (T1ce): highlights areas with a broken blood-brain barrier. Useful for identifying active tumor regions.
- Fluid Attenuated Inversion Recovery (FLAIR): used to visualize areas with pathological changes such as edemas and specific areas of the tumor.
- Binary segmentation: labeling of pixels as tumor (1) or non-tumor (0) by experts.
The data set is divided in training and validation sets (994 images) y testing set (257 images). The resolution is 240 x 240 pixels..
The DeepLab models are a type of deep learning architecture dedicated to semantic image segmentation, that is, assigning a class to each pixel of an image. They are characterized by the use of dilated convolutions, which allow the model to increase its field of view without increasing the number of parameters by introducing gaps between the pixels of the convolution mask. Versions v2 and v3 of this model also include what is known as Atrous Spatial Pyramid Pooling (ASPP), which consists of applying different dilated convolutions with varying dilation rates, enabling the capture of information at different scales. It also includes what is usually known as encoder-decoder structure which consists in the union of two structures: an encoder, which encodes the image by extracting the most important features; and a decoder, which allows obtaining the original image dimensions but from the features extracted by the encoder.
UNet models are a series of fully convolutional architectures with an encoder-decoder structure developed for biomedical image segmentation. They are characterized for using information from the encoder in the decoder through what are called skip connections. The encoder-decoder structure, along with these skip connections, gives the model its U-shape, from which it gets its name.
The AGResUNet model adds two additional elements to the structure:
- Residual connections: Typical of the ResNet model. The convolutional blocks of the UNet are replaced with residual blocks, which include information from the input image at the output of the block.
- Attention gates: Typical of the attention mechanisms of transformers. Attention gates are added during the UpSampling process of the decoder, allowing the network to focus on important regions of the MRI images.
The main libraries are:
- Pytorch = 2.5.1
- Numpy = 2.2.1
Additional dependencies are listed in the file requirement.txt
Run training.py to start training the models and visualizing the results.
For 20 epochs the metrics obtained for both models are the following:
| Models | Accuracy | Precision | Recall | Dice | IoU |
|---|---|---|---|---|---|
| DeepLabV3 | 0.991 | 0.864 | 0.914 | 0.872 | 0.799 |
| AGResUNet | 0.995 | 0.937 | 0.937 | 0.916 | 0.881 |
The results are specially satisfactory with AGResUNet. This is also shown in the following prediction:
Oktay, Ozan, Jo Schlemper, Loic Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, et al. 2018. “Attention u-Net: Learning Where to Look for the Pancreas,” April. https://doi.org/10.48550/arXiv.1804.03999.
Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. 2015. “U-Net: Convolutional Networks for Biomedical Image Segmentation.” https://arxiv.org/abs/1505.04597.
Siddique, Nahian, Sidike Paheding, Colin P. Elkin, and Vijay Devabhaktuni. 2021. “U-Net and Its Variants for Medical Image Segmentation: A Review of Theory and Applications.” IEEE Access 9: 82031–57. https://doi.org/10.1109/access.2021.3086020.
Zhang, Jianxin, Zongkang Jiang, Jing Dong, Yaqing Hou, and Bin Liu. 2020. “Attention Gate ResU-Net for Automatic MRI Brain Tumor Segmentation.” IEEE Access PP (March): 1–1. https://doi.org/10.1109/ACCESS.2020.2983075.
Zhang, Zhengxin, and Qingjie Liu. 2017. “Road Extraction by Deep Residual u-Net.” IEEE Geoscience and Remote Sensing Letters PP (November). https://doi.org/10.1109/LGRS.2018.2802944.