Image-Based Crack Detection Methods

After preprocessing of images, a multiple directional non-minimum suppression (MDNMS) method is applied for crack detection. A linear SVM classifier is used to differentiate between various pavements across Spain to determine optimal parameters for crack detection. The performance of the crack detection method is improved by adapting parameters specific to the pavement.

crack detection;machine learning;artificial intelligence;image processing

1. Introduction

Annually, millions of dollars are spent to acquire various tools and assets to carry out defect detection from key infrastructure which includes roads, bridges, buildings, and water bodies [1]. Civil structures such as roads, bridges, buildings, and pavements are often exposed to extreme physical stress which may be caused by natural disasters like earthquakes, catastrophic incidents like blasts or daily usage. Such incidents can either cause a complete collapse of the structure or may lead to physical damage that is often represented in the form of cracks. Usually, cracks emerge at a microscopic level on the surface of the infrastructure component [2]. These cracks make the component weak, reduce its loading capacity and lead to discontinuities on the surface [3][4][5][3,4,5]. If such cracks are detected at an early stage, further damage can be reduced [6]. Undetected cracks can however spread through the surface and may lead to the complete collapse of the structure, resulting in fatalities, injuries, and financial loss. Manual methods of crack detection involve experts who examine the component visually and the use of specific tools to identify any deficiency in the component [6]. However, this method is tedious, labour extensive and prone to human error. Automatic crack detection deals with using technologies to identify cracks from infrastructures. The level of degradation can be determined by analyzing the length, width, depth and severity of a crack. These measures can be used to make decisions regarding the classification of the crack, durability of the structure and its usage [7]. Using the traditional inspection procedures which involve manual inspection, it is very time-consuming to determine the crack measures which make it difficult to make inference regarding the level of degradation. Hence, for a quick, effective, and reliable damage assessment, the crack detection process must be automated to replace the manual defect inspection methods. Some testing methods like laser, infrared, thermal, radiographic, and thermal testing approaches have been used in the past to automate the process of crack detection [8][9][10][8,9,10]. However, more recently, there has been an increasing trend of using image-based methods for detecting cracks. These methods involve capturing images of the target component and analyzing them programmatically to find and classify cracks. Such methods are fast, less expensive, and robust. The methods can be categorized into two types namely as image processing and machine learning. The image processing methods do not require a model training process and involve the use of filters, morphological analysis, statistical methods, and percolation techniques for the detection of crack [11][12][11,12]. On the other hand, the machine learning process involves the collection of a dataset of images, which are supplied to the selected machine learning model for training. Such methods may involve image processing steps for preprocessing and noise removal, but the crack detection task is done by the trained machine learning model [13].

Figure 1 shows the basic architecture of an image processing-based method for crack detection. First using a camera or any other imaging mechanism, high-resolution images of the target component are collected. The images are then preprocessed which involves using filters, segmentation and other approaches to remove noise and shadows from the image. The image may be converted to grayscale or binary form if required by the specific crack detection method being used. The resultant image is applied to the crack detection procedure which uses image processing techniques like edge detection, segmentation, or pixel analysis to highlight or segment the cracked part in the image [14]. Parameter estimation involves calculating the specific properties of the detected crack such as its length, width, depth and density. Such measures help in making decisions regarding the severity of a crack.

Figure 1. Image Processing Methods for Crack Detection.

The basic steps to build a machine learning model for crack detection are depicted in Figure 2 . In the first step, a dataset must be collected showing surface cracks, which are to be detected using the machine learning model. Previously, a study led by Lin et al. used 30,000 low-resolution images for training [15]. The images are preprocessed using image processing techniques to reduce noise, remove shadows and adjust other properties such as size and brightness of images. The cracks in these images then undergo pixel-wise annotation or labelling, where the defected pixels are annotated in the image. This step can be performed manually or using a labelling tool. One such example of labelling is to set crack pixels as white or “1” in the image while the remaining pixels will be set as black or “0”. After this step, a machine learning model needs to be selected, which is to be used for crack detection. In past studies various machine learning models such as support vector machines (SVM), CNN, and decision trees have been used for crack detection [16]. A cost/loss optimization function is then formulated to minimize the loss or cost of training the model. A weighted cross-entropy loss function can be used for this purpose [17][18][19][20][17,18,19,20]. The designed model will then be trained using the set of annotated images collected in the dataset. After training the model, a new set of images will be applied to the model to see if the model successfully classifies the cracked regions in the image.

Figure 2. Machine Learning Methods for Crack Detection.

In this paper, we present a review of the image processing and machine learning-based methods for crack detection which have been proposed over the last decade. For this purpose, 30 research articles from journals and top tier conferences were retrieved and the respective crack detection methods were analyzed concerning the crack detection technique proposed, its features, performance, dataset details and the specific component to which the method is applicable. The results of each method are documented along with the corresponding limitations. A comparative analysis of these methods is conducted to highlight the most promising methods for automatic crack detection. The paper is organized as follows: Section 2 presents the methodology followed to collect data for the study. The article retrieval and screening process are explained in detail in this section. Section 3 presents the result of the study, discussing the crack detection techniques proposed in the articles and highlighting the strengths and limitations of each method. Section 4 discusses the results, analyzing the features of the methods and presenting a general picture of the current advancement in this domain. Section 5 summarizes the goals, outcomes, and achievements of this research along with the prospects.

2. Materials and Methods

To retrieve the research articles for this study, the chosen search engines were Scopus, Google Scholar, Science Direct, Elsevier, and Springer for finding the latest developments and interdisciplinary research in the field. The next step was to formulate a set of queries to be used in each of the search engines to retrieve the articles. The major aim was to fully exhaust the search database and retrieve a maximum number of articles matching the domain of interest. We used three categories of terms representing the subdomains, to extract a variety of research articles. After entering the search queries, a set of articles ranked based on their relevance were retrieved. The first category of phrases was formulated to retrieve articles that proposed flood prediction models using image processing technologies that utilised multispectral sensors. The phrases were formed by using keywords related to flood prediction which include “flood prediction”, “flood risk analysis” and “flood hazard mapping” along with phrases like “image processing” and “artificial intelligence”. The second category of terms was formulated to retrieve articles that proposed flood prediction methods using these technologies. For this purpose, we used flood prediction keywords along with the keyword “edge detection”, “mining patterns from images, Synthetic Aperture Radar, and “Image-based flood Alarm model”.

After the first phase based on article retrieval, the articles were passed through a screening phase to further narrow down the selection criteria. Four assessment criteria were defined to evaluate the articles: (1) No duplicates (2) Time interval: 2010–2021 (3) Document type: research article, abstract, book chapter (4) English language only

The next step was to specify the resource from where the research articles are to be retrieved. For this purpose, recent research articles from a wide range of journals and conferences were retrieved. Search results were obtained for each specific search phrase entered on the search engine. Screening criteria were selected based on the time frame relevance to the topic of study, and authenticity of websites for article retrieval.

The articles meeting the specified criteria for screening were downloaded. A total of 107 articles were retrieved at this stage. After this step, each article was carefully studied so that only relevant articles were kept for the research. This involved reading the abstract, methodology and results of each article. After the detailed content analysis, the articles not found relevant to the defined categories were discarded. At the end of this process, a total of 30 research articles were finally selected for the study. These articles met all the assessment criteria and were found relevant to the research questions posed in this study.

3. Analysis and CDiscurrent Insightsssion

DIn this section, we present an analysifferent arts of the crack detection studies reviewed in this paper based on the functions they have covered, the source of the dataset, the domain to which the method is applicable and the performance of the method.

All the reviewed articles were reviewed in proposed crack detection methods. However, a few of them also focused on crack classification, which deals with categorizing the crack according to its type e.g., transversal or longitudinal cracks, and sealed cracks. Many articles performed measurements of the detected crack. For example, various methods were proposed to measure the crack length, width, depth and density [34,40,41,42]. Hence, crack classification and measurement are some important problems the entrat have been tackled in several studies giving them an edge over studies that just focused on the crack detection task. Figure 7 shows that most studies (16) focused on crack detection only, while 10 focused onof them dealt with both crack detection and measurement. Just four studies proposed methods for classifying crack class after detecting them. Hence, this analysis points towards the need for further development in the classification methods for cracks. Figure 7 presents the distribution of the articles based on their features or the functionality they have provided.

Different articles were reviewed in the study focused on crack classification. Table 13 shows the crack types identified by researchers while categorization of the detected crack. This table illustrates that all the articles proposed some unique categories to classify cracks. Longitudinal (caused due to late or shallow saw cutting, inadequate base support, and built-in warping and curling) and transversal cracks (extends along the cross-section of structure, usually perpendicular to its centerline) have been tackled in two studies [6][21][6,23] while a new crack type called “sealed crack” was proposed in [20]. The authors of [15] classified the cracks based on their dimensions , hence concluded that thousands of types exist for the cracks detected in their study.

Table 1. Studies focusing on crack classification.
Crack TypesRef.
Longitudinal cracks, transversal cracks or miscellaneous.[6]
Types based on dimensions[15]
Crack, sealed crack[20]
Transverse cracks, longitudinal cracks and alligator cracks.[22]

Each method reviewed in this study is designed to detect cracks from a specific element or component. In this review, it is observed that a large majority of papers focused on detecting cracks from pavements ( Figure 39 ). A large portion of articles focused on civil structures in general. Hence, these methods can detect cracks from any infrastructure element like roads, bridges, walls and pavements. Crack detection from other materials such as leather, steel and nuclear power plant components are also considered in three of the studies [2][17][19][2,17,19]. However, the rest of the articles only focused on infrastructure components like subway tunnels, roads, bridges, pavements and walls. Hence, this points towards the need to focus on developing methods for crack detection which can be applied to other elements as well such as leather, steel, textiles and other industrial products.

Figure 3. The domain of the Crack Detection Techniques.

34. Gaps and Challenges

Crack detection is a challenging task as cracks have an irregular form with no specific shape or size. Hence, no pre-built method can be used to recognize a crack or distinguish it from the background texture and noise. In this paper, we reviewed crack detection methods belonging to two domains, namely image processing and machine learning. By examining the methodology presented in the articles and the limitations, we can infer that both these domains pose specific challenges when it comes to accurate crack detection from any component. Crack classification has been rarely addressed in the articles. The type of the detected crack facilitates the inspection process by giving important indications regarding the nature of the crack, its cause and its severity. More research needs to be focused on developing methods to classify the detected crack so that the system can recognize the type of the crack and the maintenance operations can be focused on dealing with the specific type of crack. Image processing methods have given satisfactory performance on the custom datasets built by the researchers. However, these methods are dependent on the lighting conditions, resolution of the images and the level of the noise present in the images [12][23][22][12,34,41]. Also, the surfaces of the concrete structures have varying textures as they are exposed to external disturbances and may not have the same texture even if they are built from the same material. Hence, they may not give as good results when a new image having a different texture, brightness, resolution or noise level is given as input. Furthermore, crack detection in transversal direction is not enough accurate as compared to measurement carried out longitudinally. this difference in directional measurement could be an issue when establishing a relationship between the width and longitude of the crack. Hence, the practical applicability of using image processing-based methods is still obscure [10][24][25][10,24,35].

On the other hand, machine learning methods also pose several limitations to the researchers. Increased processing time has been observed in many methods [26][70]. Many methods require manual parameter setting of the model which limits the full automation of the crack detection method [24]. Reliance on GPU is another limitation as some methods do not work as efficiently without it due to a large number of images in the dataset [17]. To avoid overfitting of the model it becomes necessary to train the model using a large dataset. These methods require extensive labelling of data images. In practical scenarios, a limited option for labelling is also available, so the acquisition of labels can be a difficult task [27][26]. Due to the difference in surface conditions, a different algorithm may be needed to accurately detect the cracks. In addition, crack detection is carried out offline so performance in real-time detection is poor. Therefore, there is a need to improve the performance of algorithms and detection accuracy in real-time. The method selected should be robust as factors such as climatic conditions may impact crack detection. Limitations in terms of the need for large datasets in order to train neural network can be overcome by empirically decomposing the fitted networks into ensembles of low-bias sub-networks, thus making use of small data sets. In addition, deep learning methods can be applied to unsupervised tasks, using a small dataset that does not require extensive labelling of data, thus reducing time and cost [28][29][48,50].

The presence of noise, shadows, blemishes and other disturbances in the images is a problem commonly faced by researchers in using both image processing and machine learning methods [5][30][5,30]. Hence, more research needs to be conducted to develop methods that can remove noise and other irregularities from images [21][31][23,71].

Top