Deep Learning in Optical Coherence Tomography Angiography: Comparison
Please note this is a comparison between Version 1 by Gabriel YANG and Version 2 by Jason Zhu.

Optical coherence tomography angiography (OCT-A) provides depth-resolved visualization of the retinal microvasculature without intravenous dye injection. It facilitates investigations of various retinal vascular diseases and glaucoma by assessment of qualitative and quantitative microvascular changes in the different retinal layers and radial peripapillary layer non-invasively, individually, and efficiently. Deep learning (DL), a subset of artificial intelligence (AI) based on deep neural networks, has been applied in OCT-A image analysis and achieved good performance for different tasks, such as image quality control, segmentation, and classification. DL technologies have further facilitated the potential implementation of OCT-A in eye clinics in an automated and efficient manner and enhanced its clinical values for detecting and evaluating various vascular retinopathies.

  • optical coherence tomography angiography
  • image quality
  • artificial intelligence

1. Introduction

Optical coherence tomography angiography (OCT-A), as the functional extension of structural optical coherence tomography (OCT), is a novel imaging modality that can provide high-resolution and depth-resolved angiographic flow images by utilizing motion contrast [1]. With the advantages of being non-invasive and more readily available, OCT-A has opened a wealth of possibilities for investigating different types of vascular damage, such as diabetic retinopathy (DR) [2], age-related macular degeneration (AMD) [3], glaucoma [4], and retinal vein occlusion (RVO) [5], as it enables the assessment of microvasculature alterations in different vascular plexuses of the retina and optic nerve head. Taking DR as an example, previous studies have validated OCT-A as an alternative to fluorescein angiography (FA) for the assessment of DR-related pathological features, such as microaneurysms, capillary non-perfusion, and neovascularization [6]. Furthermore, quantitative OCT-A metrics, such as vessel density and foveal avascular zone (FAZ) area, have been correlated with the severity of DR and visual acuity (VA) [7][8][9][7,8,9]. Longitudinal studies found that these quantitative metrics can help to predict DR progression and diabetic macular edema (DME) development [10][11][10,11]. With its reliable capacity for disease detection and prediction, the uptake of OCT-A in clinics has been sustainably growing.

2. Deep Learning-Based Algorithms for OCT-A Image Quality Control

2.1. Image Quality Grading

The generation of artifacts during image acquisition is an inherent challenge for any clinical imaging modalities, including OCT-A. There are different types of artifacts presenting in OCT-A images, and the presence of artifacts could impede image interpretation both qualitatively and quantitatively [12][13][22,23]. In most previous studies, image quality grading was performed manually. However, it is a labor-intensive, time-consuming, and resource-demanding task, which has been a significant limitation and barrier to the application of OCT-A in clinical settings. Notably, current research has shown the promise of DL-based automated image quality assessment. For example, Lauermann et al. [14][24] developed a multilayer convolutional neural network (CNN) for classifying foveal-center 3 × 3 mm2 superficial capillary plexus (SCP) OCT-A images as either sufficient or insufficient quality. The developed network was trained by a total of 160 SCP OCT-A images (sufficient group: 80; insufficient group: 80) and tested on 40 unseen images. The proposed network attained a training accuracy of 97% and validation accuracy of 100% for classifying the images into the binary classification. Yang et al. [15][25] developed a multitask DL network to assess both 3 × 3 mm2 SCP and deep capillary plexus (DCP) OCT-A images. By using more than 3500 SCP and DCP OCT-A images, respectively, for training, and another 480 SCP and DCP OCT-A images, respectively, for testing, they reported the DL network achieved areas under the receiver operating characteristic curves (AUROCs) above 0.982 for the gradability task, and AUROCs above 0.973 for the measurability task for both SCP and DCP OCT-A images derived from two types of OCT-A devices. Likewise, in order to fulfill the need for selecting qualified images for different settings, Dhodapkar et al. [16][26] trained two separate networks based on 8 × 8 mm2 SCP OCT-A images to classify high-quality images, which were for research use, and low-quality images, which should be excluded. They reported AUROCs above 0.97 for both networks. Remarkably, the developed networks were further tested on 6 × 6 mm2 SCP OCT-A images with good results (AUROCs > 0.85).

2.2. Image Reconstruction

The 3 × 3 mm2 scan is the most commonly used scanning protocol in recent OCT-A studies as it preserves a higher resolution than other wider field scans (e.g., 6 × 6 mm2 scan) for studying microvascular changes. However, the interpretation of microvascular alteration should not be limited to a 3 × 3 mm2 area as the pathological changes can also manifest elsewhere [17][27]. Therefore, it is conceivable that both wider field of view and higher resolution should be well incorporated to better address the needs of the clinical assessment. Notably, Gao et al. [18][28] proposed a DL-based angiogram reconstruction network for reconstructing low-resolution 6 × 6 mm2 superficial vascular complex (SVC) OCT-A images. In the experiment, they reported that the reconstructed images presented reduced noise and enhanced connectivity when compared to the original ones. They also concluded that the proposed network did not generate false flow signal at realistic noise intensities during image reconstruction. Later on, they further developed another reconstruction network for 6 × 6 mm2 intermediate capillary plexus (ICP) and DCP OCT-A images [19][29]. Their results indicated that the reconstruction network also applied well to the ICP and DCP as the newly developed model significantly reduced noise intensity and improved vascular connectivity without generating false flow signal. Zhang et al. [20][30] proposed a frequency-aware inverse-consistent generative adversarial network to improve the resolution of 6 × 6 mm2 SCP OCT-A images by using unpaired 3 × 3 mm2 and 6 × 6 mm2 images. By enabling the frequency transformations to refine the high-frequency information while retaining low-frequency information, their model successfully reconstructed the OCT-A images and outperformed other state-of-the-art methods in terms of peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and normalized mutual information (NMI).

3. Deep Learning-Based Algorithms for OCT-A Image Segmentation

3.1. Foveal Avascular Zone Area

The FAZ is a region surrounding the fovea which is devoid of retinal capillaries. It has been one of the most reported metrics ever since the invention of OCT-A. The literature has related the size and the intactness of the FAZ to the severity and progression of many retinal diseases [9], as well as the deterioration of VA [21][22][31,32]. Since both the shape and intactness of the FAZ carry important clinical implications, many studies have developed algorithms for automated segmentation and measurement. For instance, by using eighty 1 × 1 mm2 OCT-A images from healthy volunteers, Prentašic et al. [23][33] trained and validated a DL network for segmenting FAZ area, attaining a maximum mean accuracy of 0.83. Mirshahi et al. [24][34] also developed a DL network for the segmentation and measurement of the FAZ. Specifically, for the FAZ segmentation task, the proposed network achieved a mean dice similarity coefficient (DSC) of 0.94 ± 0.04 when compared to the results produced by the device’s built-in software; while for the FAZ measurement task, among the healthy subjects, excellent agreements were reported between the device-based and manual measurement (95% limits of agreement (LoAs) of −0.005 to 0.026 mm2) as well as between the DL and manual measurements (95% LoAs of 0.000 to 0.009 mm2). Similarly, Guo et al. [25][35] also proposed a DL network with an encoder–decoder architecture to automatically perform the segmentation and quantification of the FAZ area in SCP under different brightness/contrast settings. They reported a maximum mean DSC of 0.976 ± 0.01 when comparing the automatic segmentation results against the ground truth, and a correlation coefficient of 0.997 between ground truth and automatic segmentation results for calculating the FAZ area.

3.2. Vessel Segmentation

Retinal vasculature is critical for the nourishment of retinal tissue to maintain the normal function of the visual pathway. Pathological alterations in vascular structure have not only been used for the detection and classification of different fundus diseases [26][36], but also been linked with systemic diseases [27][37]. Studies have implemented OCT-A with DL to facilitate an automatic vessel segmentation. Ma et al. [28][38] introduced a novel split-based coarse-to-fine method for vessel segmentation in both SVC and deep vascular complex (DVC) OCT-A images. The network consisted of a segmentation module to first produce the preliminary confidence vessel map, and then further used a consecutive refining model to refine and optimize the contour of the microvasculature. The model outperformed both traditional and other state-of-the-art DL models by achieving the highest AUROC, accuracy, kappa score, and dice coefficient for segmenting the vessel. Liu et al. [29][39] proposed a disentangled representation DL model to facilitate the vessel segmentation across different OCT-A devices. By enabling the DL model to learn the disentanglement of the anatomical component (the microvasculature in images) and the local contrast component (the image background noise diversities among different OCT-A devices), their model demonstrated good performance for OCT-A vessel segmentation among different devices. Furthermore, Guo et al. [30][40] proposed a 3D CNN to segment vessels in SVP, ICP, and DCP directly from the angiographic volumetric data. Notably, their model was able to convert the data from three dimensions to two dimensions by using a custom projection module for connecting both the retinal layer segmentation and vasculature segmentation modules. The network achieved F1 score > 0.90 in SVP, >0.70 in ICP, and >0.78 in DCP for the vessel segmentation.

3.3. Non-Perfusion Area

Non-perfusion area (NPA) is a quantitative biomarker useful for characterizing retinal ischemia. The severity of retinal ischemia has been reported to not only impact anatomic and functional outcomes [7], but also associate with the clinical course and responsiveness to treatment [31][32][41,42]. With the attempt to facilitate and compare the efficiency of automated detection of NPA, Nagasato et al. [33][43] conducted research to compare the diagnostic ability among the DNN, support vector machines (SVMs), and seven ophthalmologists for distinguishing RVO with NPA from normal controls in both SCP and DCP OCT-A images. They reported that the performance of the DNN was significantly better than that of SVMs in mean AUROC, sensitivity, specificity, and average required time (all p < 0.01), as well as outperformed ophthalmologists in terms of AUROC and specificity (all p < 0.01). Guo et al. [34][44] applied a DL-based algorithm for the detection and quantification of NPA across eyes of healthy subjects compared to patients with different DR severities on widefield OCT-A images constructed by a montage of nasal, macular, and temporal scans. The algorithm showed good agreement with manual delineation (average F1 score > 0.78) for NPA segmentation across all scans. In addition, they demonstrated that NPA measured in the montage widefield images correlated with both VA and DR severity significantly, and its diagnostic accuracies for distinguishing any DR, referable DR, and severe DR were even superior to NPA measured from the traditional macular scan (all p < 0.0001).

3.4. Neovascularization

The accurate identification and segmentation of choroidal neovascularization (CNV) are essential for the diagnosis and management of chorioretinopathies [35][45], such as exudative AMD and myopic CNV, as they require urgent referral for timely intervention. Wang et al. [36][46] developed a fully automated algorithm for the detection and segmentation of CNV in OCT-A images, with a total of 1676 scan including follow-up scans used for training and testing. During testing, their algorithm attained a 100% sensitivity and 95% specificity for differentiating CNV cases from the non-CNV controls. Additionally, an intersection over union (IoU) of 0.88 was reported between the human graders and the algorithm for the segmenting the CNV membrane. Likewise, Thakoor et al. [37][47] developed a hybrid 3D (OCT-A volume scans)–2D (OCT-B scans) CNN to facilitate a multiclass categorical AMD classification with a total of 346 eyes (97 non-AMD, 169 non-neovascular AMD, and 80 neovascular AMD) enrolled for training, validation, and testing. They reported an accuracy up to 77.8% for classifying different stages of AMD, illustrating the tremendous potential of DL algorithms concatenating multiple imaging modalities to expedite the screening for early- and late-stage AMD patients.

4. Deep Learning-Based Algorithms for OCT-A Image Classification

4.1. The Classification of Artery and Vein

The differentiation of artery–vein (AV) analysis can not only provide valuable information for retinal diseases, but also new insights on systemic diseases. For example, the narrowing of retinal arteriole has been reported to associate with hypertension, while both arteriolar and venular tortuosity have been shown to relate to DR progression [38][39][40][41][42][48,49,50,51,52]. As OCT-A images are extensively reported to reveal subtle microvascular changes, recent studies have combined both DL and OCT-A to facilitate the AV classification. For example, Alam et al. [43][53] developed a fully CNN based on modified U-shaped architecture to differentiate arteries and veins in OCT-A images, with a transfer learning process also being integrated to compensate for a limited dataset. They reported their algorithm achieved an average accuracy of 86.75%, a mean IoU of 70.72%, and an F1 score of 82.81% on the test data, of which outperformed other state-of-the-art models, as well as the model without using transfer learning. Gao et al. [44][54] further proposed a CNN for AV classification in montaged widefield OCT-A images. Specifically, they used only 6 × 6 mm2 OCT-A images from the nasal, macula, and temporal area for training and validating the algorithm, and testing on both 6 × 6 mm2 and 9 × 9 mm2 images. The proposed algorithm attained an F1 score > 94.1% and IoU > 89.2% for the AV classification across two devices and different scan sizes.

4.2. The Classification of Diabetic Retinopathy Severity

In addition to retinal photograph-based algorithms, DL has also been making remarkable breakthroughs with several OCT-A-based algorithms also being built to enhance DR classification and management. For example, by enlisting diabetic eyes ranging from no DR to different severities of DR, Ryu et al. [45][55] developed a fully automated classification algorithm to identify the onset and referable status of DR in OCT-A images. The proposed algorithms achieved AUROCs above 0.93, and accuracies, sensitivities, and specificities all above 85% for detecting the onset of DR and referable DR, both in the internal validation and external testing. Le et al. [46][56] developed a CNN implemented with a transfer learning process for retraining the model to perform a trinary classification, namely, healthy, diabetic mellitus (DM) but with the absence of DR, and with the presence of DR, on OCT-A images. Their model also attained good performance with a cross-validation accuracy of 87.27%, sensitively of 83.76%, and specificity of 90.82% for differentiating the trinary outcomes, and AUROCs all above 0.97 across the binary classification among healthy, DM but with the absence of DR, and with the presence of DR. In order to utilize information both from OCT and OCT-A, Zang et al. [47][57] developed an automated model to produce three classification levels to facilitate the clinical diagnosis of different stages of DR. The first level of the model was designed to classify non-referable and referable DR; the second level was to differentiate no DR, non-proliferative DR (NPDR), and proliferative DR (PDR); and the third level was to perform a full DR classification, namely no DR, mild and moderate NPDR, severe NPDR, and PDR. They reported overall classification accuracies of 95.7%, 85.0%, and 71.0%, respectively, for the three classification levels.

4.3. The Classification of the Presence or Absence of Diabetic Macular Ischemia

Previous studies have reported that the severity of macular ischemia is associated with irreversible visual deterioration, as well as the treatment response following anti-VEGF therapy in eyes with concomitant DME [32][42]. Yang et al. [15][25] developed a multitask DL system to first assess the image quality, and then classify the presence or absence of diabetic macular ischemia (DMI) in both SCP and DCP OCT-A images. In order to train the model to perform “DMI assessment” based on the ETDRS protocols, they defined the presence of DMI as OCT-A images exhibiting disruption of FAZ and/or additional areas of capillary nonperfusion in the macula, while the absence of DMI was classified as images exhibiting intact FAZ outline and normal distribution of vasculature. Their model achieved AUROCs > 0.939 and areas under the precision–recall curves (AUPRCs) > 0.899 for the DMI assessment across three external validation datasets compromising two different types of OCT-A devices [48][58].

4.4. The Classification of Healthy Eyes and Glaucoma

Glaucoma is among the leading causes of irreversible vision loss globally. Earlier OCT-A studies have revealed that in comparison to healthy eyes, glaucoma eyes showed significant attenuations in optic disc perfusion and peripapillary vessel density [49][50][59,60]. These alterations were not only associated with worse structural and functional glaucomatous measurements, but also provide predictive values for glaucoma progression [4][50][4,60]. Of note, Bowd et al. has compared a CNN to conventional ML, i.e., gradient-boosting classifiers (GBCs), for classifying healthy and glaucomatous eyes based on OCT-A metrics [51][61]. Specifically, the DL model was trained and tested on 4.5 × 4.5 mm2 radial peripapillary capillary OCT-A optic nerve head (ONH) images, and further compared with separate GBC models trained and tested on standard OCT-A and OCT measurements. The adjusted AUPRC for classifying healthy and glaucoma eyes was significantly improved by using the DL-based CNN analysis of OCT-A metrics in comparison to the conventional GBC analysis (p ≤ 0.01). On the other hand, instead of comparing DL and ML models, Schottenhamml et al. [52][62] demonstrated that by training the CNN using 3 × 3 mm2 OCT-A images of different retinal projections (of the whole retina, SVC, ICP, and DCP), the CNN performed similarly well to, or even better than, the handcrafted methods for distinguishing glaucomatous eyes from healthy controls, especially when using features from the whole retina projection and the SVC projection.
Video Production Service