The prevalence of colorectal cancer is increasing in Asia, including Japan, and worldwide ^[1][2][1,2]. It generally develops according to the adenoma-carcinoma sequence [3], and the early colonoscopy detection, resection, and treatment of colorectal adenoma have been shown to reduce not only the prevalence of colorectal cancer but also its mortality rate ^{[4][5][6][7][8][9][10]}[4,5,6,7,8,9,10]. Accordingly, in Japan, it is recommended that individuals aged 40 years or over who test positive for fecal occult blood in health checkups undergo further investigation by total colonoscopy (TCS), and in the United States (US), screening by TCS for all individuals aged 50 years or over who have not previously undergone endoscopic examination is recommended as part of a national project [4]. However, post-colonoscopy colorectal cancer (PCCRC) that develops after TSC is becoming a problem. Cases of PCCRC resulting from a lesion having been overlooked or incompletely resected have been reported [11], and laterally spreading tumors (LSTs) in the right hemicolon and rapidly-growing de novo cancers are said to be particularly easy to overlook. One study has found that an adenoma detection rate (ADR) of <20% significantly increases the risk of PCCRC compared with an ADR of ≥20% [12], and the importance of the ADR as a quality indicator from the viewpoint of picking up lesions has been noted [13]. This situation highlights the significance of aiming for TCS with a low miss rate, and devices such as extra-wide-angle endoscopes and tip hoods are now being used in the attempt to detect lesions in difficult-to-see blind spots. It has also been suggested that image-enhanced endoscopy, such as high-definition endoscopy and chromoendoscopy, may also be useful for the identification of superficial or depressed lesions that are difficult to see even when they are located within the field of view.

Today, there are increasing expectations for the use of computer-aided diagnosis/detection (CAD) systems that utilize AI. CAD is broadly classified into two categories depending on its purpose: computer-aided detection (CADe) is used to pick up the location of candidate lesions based on the analysis results, and computer-aided detection (CADx) is used to present information on qualitative diagnosis. If sophisticated CAD systems can be developed, in theory, they could provide high-level diagnostic performance irrespective of the level of skill of the endoscopist, and the development of such CAD is thus impatiently awaited.

2. CADe (Computer-Aided Detection)

Computer-aided detection is the use of AI as diagnostic assistance for picking up lesions. As mentioned above, a high ADR is an accepted quality indicator. In one study, a 1% increase in ADR resulted in a 3% decrease in PCCRC and a 5% decrease in fatal PCCRC [13]. Because preventing lesions from being overlooked has been prioritized, it is CADe research that is particularly advanced in AI research in the colorectal field. Methods of automatically detecting polyps using a range of different imaging feature quantities (such as edge detection, texture analysis, and energy mapping) had been under investigation since the turn of the millennium, but none of these had reliable detection rates of ≥90%, and limitations on computational capacity meant that none were successful in providing a real-time diagnosis. These methods had the further disadvantage that they only responded to lesion morphology. However, this situation was transformed by the advent of deep learning (DL) in the second decade of the century. By adding temporal elements to DL, Misawa et al. developed a convolutional neural network capable of real-time polyp detection with 90.0% sensitivity and 63.3% specificity [14]. Urban et al. also used a large number of static endoscopic images as training images and succeeded in detecting polyps with very high rates of 93.0% sensitivity and 93.0% specificity [15]. Subsequently, AI research has proceeded by constructing DL algorithms using retrospectively assembled endoscopic test images, and the performance of these AI systems in clinical settings is currently being analyzed in six prospective randomized clinical trials. Five of these six trials have the ADR as the primary endpoint ^{[16][17][18][19][20]}[16,17,18,19,20]. In the other one, the primary endpoint is the adenoma miss rate (AMR) [21]. The most important finding is that in all the trials focusing on a comparison of the ADR, a significant increase in this parameter is evident. To summarize the data from these trials, the ADR increased by 6–15.2% depending on the investigator’s skill and enrollment criteria. In one multicenter joint study using AI, a rate of 40.4% in the control group rose to 54.8% in the CADe group [16]. This suggests that the risk of PCCRC is decreased by assuring a certain ADR [12], and given that the ADR is inversely correlated with mortality [13], the widespread adoption of CAD technology in clinical settings would be highly advantageous. According to a more detailed analysis of the randomized clinical trials taking the ADR as the primary endpoint, the detection rate of small adenomas measuring ≤5 mm increased significantly in all these trials [22]. In just one of those trials, the detection rate of adenomas measuring 6–9 mm also rose [16]. Despite the increase in the adenoma detection rate, however, in all the studies there was no significant difference in the scope withdrawal time (excluding polyp resection time) depending on whether CADe support was used or not, a result that is encouraging for its use in actual clinical settings where time is limited. These benefits suggest that the tendency in recent years for the ADR to increase [23] will probably continue and that an increasing number of institutions can be expected to introduce CADe in the future.

3. CADx (Computer-Aided Diagnosis)

Computer-aided diagnosis is the use of computer programs for the qualitative diagnosis of lesions and the evaluation of disease activity. Unlike CADe, in which observations are made under normal white light, in CADx not only white-light endoscopy ^[24][25][24,25] but also a wide range of other modalities, including magnifying narrow-band imaging (NBI) ^{[26][27][28][29][30][31]}[26,27,28,29,30,31], linked-color imaging (LCI) [32], blue-light imaging (BLI) ^[33][34][35][33,34,35], magnifying chromoendoscopy [36], ultra-high-magnification endoscopy ^{[37][38][39][40][41]}[37,38,39,40,41], confocal laser endoscopy ^[42][43][42,43], autofluorescence spectroscopy ^[44][45][44,45], and autofluorescence imaging (AFI) ^[46][47][48][46,47,48]. Of these, magnifying NBI has been best studied. The fact that its magnified observations improve diagnostic capability and that time-consuming procedures such as pigment spraying are not required may be an advantage that makes it easy to use in everyday clinical practice. Triggered by the start of CADx research on the evaluation of pit patterns by chromoendoscopy, which began after the turn of the millennium ^[36][49][36,49], several research groups have published reports targeting magnified NBI images since 2010 [27]. Some of those studies have come close to achieving practical implementation of real-time diagnosis ^[26][36][26,36]. The diagnostic algorithms used are mainly the methods of using machine learning (such as a support vector machine, neural network, or k-NN classifier) to conduct learning and classification by using a large number of characteristics derived from image filters and texture analysis. In a prospective clinical trial conducted by Kominami et al. [26], the researcheuthors validated real-time CADx for 118 lesions in 41 patients, achieving 93.3% sensitivity, and 93.3% specificity. Although that researchstudy was small, it is the only prospective researchstudy in theis research area. Since then, DL has garnered attention and become predominant in this field of research. Conventional methods of machine learning require high-level technical skills and knowledge of information engineering in the process of numerically converting endoscopic characteristics, and this constitutes a high hurdle for its development. With DL, however, the process of numerical conversion of these characteristics is simplified, greatly lowering the hurdle for its development; as a result, it has come to be frequently used. Chen et al. [31] and Byrne et al. [30] both succeeded in developing CADx systems using DL that distinguish between tumors and other lesions with better than 90% sensitivity while using the comparatively small number of 3000 images and 300 videos, respectively, as training images based on NBI imaging. This accuracy could be further improved by using a larger number of training cases, and its validation in future prospective studies is awaited. The majority of polyps discovered in screening colonoscopy are small lesions measuring ≤5 mm, and it is rare for these small polyps to exhibit neoplastic growth and malignant transformation ^[50][51][50,51]. Polypectomy and pathological investigations themselves may therefore entail a disproportionate burden in terms of cost and effort [52], and a CADx system with high diagnostic performance might enable the choice between “resect and discard” and “diagnose and leave” to be made from the viewpoints of time and cost as well as complications. The target of most studies in this field is to distinguish between neoplastic and non-neoplastic lesions, but Tamai et al. [53] used 121 colorectal lesions to develop a CADx system to distinguish those with deep submucosal invasion (T1b cancer), reporting that it distinguished them with 83.9% sensitivity and 82.6% specificity. Colorectal T1b cancer is not easy to distinguish, and as the diagnostic accuracy of clinicians is known to be under 80% [54], the practical implementation of this CAD system could be highly beneficial in clinical practice.