- The success of radiotherapy for lung cancer depends on the control of radiation exposure to organs at risk (OARs) such as normal lungs, esophagus, spinal cord and heart, etc. Hence, accurate normal tissue delineation is crucial for the outcome of radiotherapy — this is currently done manually by clinicians on CT images which tedious, time consuming, and laborious.
- Atlas based method, which is available in several commercial products, registers atlas template that contain pre-contoured structures. However, computation cost, variability across patients, and unpredictability of tumor shape makes such deformable image registration inefficient.
- In this paper, authors employ the GAN strategy, with U-Net as a generator (generates image segmentation maps) and FCN (Fully Convolutional Network) as a discriminator (discriminates between predicted mask vs. manually delineated mask).
- First, 3-label-based segmentation model simultaneously segment three organs, out of five, of similar sizes: the heart, left lung and right lung. The segmentation model is 2.5D end-to-end patch-based GAN model which takes four continuous slices of CT images (512 x 512 x 4). *Spinal cord and esophagus are much smaller than heart and lungs, and cannot be segmented simultaneously.
- Esophagus and spinal cord segmentation are trained separately with 3D GAN on cropped region of interest patches (ROI). These ROIs are obtained based on the relative position of the esophagus and spinal cord to the lungs. Center of esophagus ROI is set as the centroid of the total lung, and the center of spinal cord ROI is set as the midpoint of the two most posterior points of left lung and right lung in the same slice. 64 x 64x 64 patches were used as inputs for both models.
- Finally, all the segmentations had their respective locations determined based on spatial information of the original CT patches. The OAR contours are reconstructed with patch fusion and refined by contour refinements.
Generative Adversarial Network:
- Why GAN? Contouring variability of manual contours results in instability of end-to-end network model. GAN model introduce extra judgment with discriminator to help generator find the optimal solutions.
- How? GAN based segmentation model consists of generator that generates segmentation mask and discriminator the discriminates between mask predicted by generator (fake) and ground truth (manual segmentation; real). The goal of generator is to maximize judge error of discriminative network, and the goal of discriminator is to decrease the judge error of differentiating the real from fake. In this paper, generator model is U-Net and discriminator is classification based FCN which output 1 x 1 x 1 variable with 1 denoting real and 0 denoting fake.
- Generator Loss The generator loss was computed as the sum of mean squared error (MSE) of the “residual” images (=residual loss) and the binary cross entropy loss of contour images (=discriminator loss) . Residual images are calculated as the element-wise multiplication of the original CT patches with the probability maps of generated contours, and the reference residual images are calculated as multiplication of CT patches and segmentation masks generated by manual contouring. Residual loss is used instead of binary cross entropy of Dice loss to account for the size difference between different labels.
- Evaluation 6 metrics were used to evaluation the performance: dice similarity coefficient (DSC), sensitivity, specificity, 95% Hausdorff distance (HD95), mean surface distance (MSD), and residual mean square deviation (RMSD). (I will be posting about evaluation metrics for segmentation tasks & its implementation soon!)
- The proposed method achieves superior segmentation accuracy on the left lung, right lung and spinal cord with respective mean DSC of 0.97, 0.97 and 0.90. Heart segmentation, due to reduced image contrast, is less straightforward and yields mean DSC of 0.87. Esophagus has the lowest contrast, thus most difficult one to delineate. The proposed method obtains 0.75 DSC.
- Comparison between segmentation performance of U-Net and U-Net GAN suggests that applying GAN scheme significantly improves the segmentation performance of U-Net.
- Comparison between the dose of OARs from 20 Lung SBRT plans based on ground truth (manual) contours and auto-contouring show no statistically significant difference in dosimetric impact.
Joon Yau Leong, Amir S. Patel, R. R. (2017). 乳鼠心肌提取 HHS Public Access. Physiology & Behavior, 176(5), 139–148. https://doi.org/10.1002/mp.13458.Automatic