A novel 3D segmentation approach for extracting retinal layers from optical coherence tomography images

Purpose: Accurate segmentation of retinal layers of the eye in 3D Optical Coherence Tomography (OCT) data provides relevant information for clinical diagnosis. This manuscript describes a 3D segmentation approach that uses an adaptive patient-specific retinal atlas, as well as an appearance model for 3D OCT data. Methods: To reconstruct the atlas of 3D retinal scan, the central area of the macula (macula mid-area) where the fovea could be clearly identified, was segmented initially. Markov Gibbs Random Field (MGRF) including intensity, spatial information, and shape of 12 retinal layers were used to segment the selected area of retinal fovea. A set of coregistered OCT scans that were gathered from 200 different individuals were used to build a 2D shape prior. This shape prior was adapted subsequently to the first order appearance and second order spatial interaction MGRF model. After segmenting the center of the macula “foveal area”, the labels and appearances of the layers that were segmented were utilized to segment the adjacent slices. The final step was repeated recursively until a 3D OCT scan of the patient was segmented. Results: This approach was tested in 50 patients with normal and with ocular pathological conditions. The segmentation was compared to a manually segmented ground truth. The results were verified by clinical retinal experts. Dice Similarity Coefficient (DSC), 95% bidirectional modified Hausdorff Distance (HD), Unsigned Mean Surface Position Error (MSPE), and Average Volume Difference (AVD) metrics were used to quantify the performance of the proposed approach. The proposed approach was proved to be more accurate than the current state-of-the-art 3D OCT approaches. Conclusions: The proposed approach has the advantage of segmenting all the 12 retinal layers rapidly and more accurately than current state-of-the-art 3D OCT approaches.