Different CNN models are used in crack segmentation methods. The effectiveness of CNNs in crack segmentation has been demonstrated in a large number of experiments.

Multi-scale images are one of the most challenging aspects of computer vision, and most networks directly use the features extracted from the backbone network to predict the target. This pioneering work [10] proposed a top-down approach to multi-scale features using the feature pyramid network (FPN). Following this idea, PANet [11] adds a bottom-up path aggregation network on top of FPN to further exploit the different information between the bottom and top-level features. NAS-FPN [12] uses neural architecture searching to design feature network topologies automatically but requires thousands of hours of GPU time and makes it difficult to interpret the network structure. BiFPN [13] adds a bi-directional scale join based on removing nodes with only one input edge and adding a residual structure to join the original junction and the output node. Each of the four network structures is shown in Figure 2.

The fusion of features at different scales achieved good results using all of the methods above, but feature fusion leads to confounding effects [14], along with bias towards local interactions. By combining the CCT module, researchers optimize features across multiple scales after feature fusion to reduce confounding effects and establish long-range dependencies.

The transformer [15] program has achieved great success in the natural language processing domain, and many researchers have applied the program to tasks related to computer vision. A pure transformer vision model (Vision Transformer, ViT) was proposed by Dosovitskiy [16], which divides, convolves, and spreads images into two-dimensional vectors to solve the transformer’s input problem in the image domain. Liu [17] proposed the Swin-Transformer, which uses a hierarchical construction method similar to that used in convolutional neural networks: chunking the feature maps to reduce computational effort and using spatially shifted bit windows to facilitate information exchange between different windows, thus improving the model’s ability to extract local information. The state-of-the-art level of ImageNet classification is achieved. Inspired by Swin-Transformer, Cao [18] proposed Swin-Unet, the first purely transformer-based U-shaped structure, which uses Swin-Transformer to replace the convolution module in U-Net. Wang [19] proposed UCTransNet, using a CTrans (channel transformer) module instead of skip connections in U-Net. The CTrans module consists of multi-scale channel cross-fusion (called CCT) and channel cross-attention (called CCA). The CCT module performs cross-fusion of multi-scale features through the transformer. The CCA module directs the fusion of the fused multi-scale features with the decoded features through attention.

The Swin-Unet algorithm requires a large amount of data to obtain better performance. It lags behind the convolutional network when small amounts of data are available. UCTransNet directly uses the features of the backbone network as input to the transformer. It does not take full advantage of the underlying detailed information. This entry uses the pure convolutional network ConvNext as the backbone to extract features that better fit small datasets. The information at different scales is better utilized using a feature fusion network.

TMCrack-Net consists of three main modules: (1) Top-down extraction of crack features using the ConvNext network as the backbone of the U-net network. (2) The AG-BiFPN module includes BiFPN and CCT, a feature fusion network, to exploit the multi-scale features of the image better. (3) An attention fusion module to better enhance useful features and suppress invalid ones. Figure 3 shows the TMCrack-Net with BiFPNet, Transforme, and FCA.

In this entry, researchers consider the crack segmentation task of mural images as a pixel-level semantic segmentation problem, where 0 represents “background pixels” and 1 represents “crack pixels”. Compared with natural images, the number of mural images is small, and the background information is complex, so the feature extraction ability of the backbone network directly affects the segmentation effect. Researchers' backbone is the ConvNext [20] network proposed by Facebook AI Research and the University of California, Berkeley, which optimizes ResNet to reach the performance limits of pure convolutional models. As a pure convolutional network, it achieves 87.8% TOP-1 accuracy on the ImageNet dataset while maintaining the simplicity and efficiency of standard convolutional networks.

ConvNext consists of downsample blocks and ConvNext blocks, repeatedly stacked. A ConvNext block consists of depthwise convolution, and ordinary Conv is combined, using the GELU function as the activation function. A downsample block consists of layer norm and 2 × 2 convolution. Figure 4 shows the structures of downsample and ConvNeXt blocks.

With the small number of mural images and complex background information, low-level information and high-level semantic information are particularly important for crack segmentation. Using only the backbone network to extract features from the images does not make good use of the channel information at different scales, and a feature fusion network needs to be added to recycle the multi-scale features, but feature fusion confuse the localization of cracks in the images. Attention mechanisms need to be used to optimize the feature pyramid, and using attention computation at each layer of the pyramid is costly. Therefore, researchers introduced the CCT [19] module. By analyzing the information provided by each element layer, CCT can automatically determine the importance of different channels, increase the weights of channels that contribute to segmentation, and suppress channels that do not. Due to the ambiguity between the features of the codecs, in this entry, researchers design an FCA module based on a self-attentive model to filter the channel information and better fuse the features between them.

UCTransNet demonstrates that not all jump-joins are beneficial for segmentation, and simple replication of features at the encoder stage can be detrimental to feature fusion, so more suitable feature fusion methods are needed to join the encoder and decoder.

In order to better utilize the features extracted from the backbone network, researchers invoke a combination of BiFPN and CCT modules instead of the original hop-join in the U-net. By fusing feature information at different scales through the BiFPN network, the input image retains a richer level of detail contained in the shallow features, better utilizes local information, and focuses on extracting crack details.

BiFPN was proposed by Google Research, based on the bi-directional scale join. It removes the nodes with only one input edge, which do not perform feature fusion and contribute little to the network, and add the residual structure to join the original junction with the output node, which can fuse more features. The BiFPN infrastructure is shown in Figure 2d.

The single-use feature fusion network with only a convolutional structure is biased toward local interaction and brings some confounding effect, which interferes with recognizing cracks. By adding an attention mechanism after BiFPN, as shown in Figure 5, the CCT module can better emphasize the global information and alleviate the confounding effect at each layer of the pyramid.