A popular method to automate AM mapping of aboveground assets is through computer vision algorithms. This task consists of identifying key utility assets from pictures. Figure 1 illustrates an example through a street-level image, showing identifiable utility-related appurtenances and the position of underground utilities at that location.
Overall, the use of AI in conjunction with AM processes includes a vast array of methods to conduct both asset identification and buried utility location. Yet, from the review, the vast majority of works focused on the detection of manholes and their use in utility mapping. This disproportionate attention precluded the applications of AM and AI techniques to other utilities, such as electricity, natural gas, and telecommunication. There is also a lack of frameworks that combine both the tasks of asset identification and utility location.
2. Terrestrial Mapping
Terrestrial mapping (TM), not including photography, is typically conducted to achieve higher levels of mapping accuracy than those attainable via AM techniques. These can include the use of geophysical investigations or intrusive measures to achieve SUE quality-level A mapping accuracy through techniques such as vacuum excavation
[12]. This method is widely used in today’s utility locating practices
[13][14]. However, intrusive exposures for mapping can be costly as it can potentially disrupt critical activities such as construction and traffic
[15], especially where utilities are within transportation right of way. The invasive technique also requires close attention to details during excavation to prevent damage to buried assets. A solution is to adopt non-intrusive sensing systems and geophysical tools such as ground-penetrating radar (GPR), metal detectors, vibro-acoustic systems, radio-frequency locators, and geoelectrical-resistive sensing methods
[16]. These sensing methods output SUE quality-level B data
[17], which has the potential to be further refined using data fusion techniques
[18][19].
However, non-intrusive sensing requires highly trained operators to assess and evaluate information from scanned data
[20]. Data processing can also be challenging, even for experts
[21]. AI techniques provide a promising solution for assisting the non-intrusive process by automatically and rapidly evaluating sensing data to support actionable decisions.
GPR-generated data can be complex to analyze. Typically, the hyperbolic signatures form by joining the discrete travel time of the electromagnetic waves as they pulse back from the targets
[22]. Information such as the object’s depth and horizontal position is estimated using the hyperbolic signature’s spread, position in radargram, and degree of resolution. Yet, the manual reading of radargrams is often challenging due to the intrinsic differences in material properties of targets and soil characteristics. Several AI tools were developed and applied to assist the interpretation of GRP-collected data, in particular for applications regarding underground utilities
[23][24][25][26].
One popular tool is the Hough transform (HT), used to extract geometrical features in images
[27]. It was used by Capineri et al.
[28] in locating the position of buried objects represented by hyperbola in GPR radargrams. The HT was also used in Al-Nuaimy et al.
[29], where the HT was combined with a backpropagating neural network to detect buried utilities and other solid objects. Al-Nuaimy et al.
[30] proposed an adaptive non-accumulated Hough transform (ANHT) to reduce the memory demand relative to the conventional HT and demonstrated that the technique could be applied to detect linear structures such as pipes and cables. Modified HT variants were proposed. For instance, Simi et al.
[31] developed the randomized HT that yielded underground utility detection accuracy greater than 85% over different ground conditions such as asphalt, grass, and concrete; Al-Nuaimy et al.
[29] used a modified HT to detect targets in a GPR radargram with a significantly low signal-to-noise ratio; and Maas et al.
[32] combined the Viola–Jones learning algorithm with the HT to separate regions of interest and extract vertices and locations of the hyperbola. Other works adopted the HT technique for recognizing, localizing, and evaluating hyperbolic indications, wave velocity, buried pipe diameters, and asset depths in radargram
[33][34][35][36].
It was argued in Chen and Cohn
[36] that some early methods enabling automatic detection in GPR hyperbolic signatures were not suitable for on-site applications due to their computation complexity and unsatisfactory detection rate resulting from insufficient training. The authors proposed a probabilistic hyperbolae mixture model based on a classification expectation-maximization algorithm and a Bayesian information criterion for model selection. Compared to other HT, their proposed model completed the utility identification task in 1.7 s, while the HT completed a similar task in 226.1 s. In Dou et al.
[37], a column-connect clustering algorithm was developed, which first discriminates targeted utilities represented as hyperbolas from the radargram background, with the algorithm yielding a 23% improvement in detection rate than for the pattern recognition technique used for a similar task in
[32]. A variant of the column-connect clustering algorithm was investigated by Rosin, P.
[38] that resulted in a 6% improvement in the model accuracy (F1-score).
Support vector machine (SVM) gained popularity in GPR data classification tasks. Zhang et al.
[39] used SVM for the automatic extraction and classification of buried landmines from a GPR radargram; El-Mahallawy et al.
[40] adopted an SVM classifier to identify underground utilities and showed 100% and 78% recognition accuracies in an ideal and noise-degraded environment, respectively; Lu et al.
[41] proposed to combine a discrete wavelet transform, fractional Fourier transform, and SVM for classifying subsurface objects material from GPR data; Terrasse et al.
[42] detected buried gas pipes using an SVM classifier and reported a 96% detection rate; Kaur et al.
[43] combined HOG with SVM to map deteriorated reinforcement bars; Noreen et al.
[44] automated buried pipe detections using HOG features clustered with an SVM algorithm; and Ozkaya et al.
[45] fed features extracted from GPR radargram images using convolutional neural networks into an SVM algorithm to detect and classify hyperbolic signatures, material, shape and soil type.
Genetic programming (GP) was employed in automating GPR data processing. Pasolli et al.
[46] proposed an automatic detection process that uses an iterative genetic algorithm to detect and locate the position of underground utilities in GPR radargrams. Kobashigawa et al.
[47] adopted GP to detect buried unexploded ordnances in GPR radargrams. The GP algorithm performance was compared against different neural networks of varying hidden layers and showed improved performance over complex radargrams. Additionally, Harkat et al.
[48] proposed to utilize a radial basis network with a multi-objective genetic algorithm to improve region classifications.
There is also a vast amount of research that focused on the use of neural networks in processing GPR radargrams. For example, Birkenfeld, S.
[49] presented a non-fully connected neural network model that identifies interfering or incomplete hyperbolas. Lei et al.
[21] proposed the faster region-based convolution neural network (R-CNN) and data augmentation to detect regions of buried objects in GPR images. Liu et al.
[50] proposed a deep learning model, termed a single shot multi-box detector, to detect hyperbolae. When compared with the faster R-CNN model adopted by Lei et al.
[21], the technique showed a higher precision rate with lower computational time. Yamaguchi et al.
[51] conducted the 3-D location of buried pipes in GPR radargrams using 3D-CNN. A 10% improvement in accuracy level was observed when compared to an SVM. The extracted positions, inclinations, and arrangement of pipes were visualized in a 3-D map using Kirchhoff migration. Liu et al.
[52] identified limitations in using hyperbolic signatures as the primary representative feature for buried utilities. The 3D GPR scan images formed by stitching parallel scan data were used as input to a 3D CNN. The proposed method produced a detection accuracy of 82.7%, compared with 69% for CNN with 2D GPR input data and 61% to 73.5% for other methods that included the AlexNet, ResNet50, VGG16 and 19, and Triplanar algorithms.
In summary, most of the reviewed literature in this section focused more on GPR because it remains the most used electromagnetic sensor for utility mapping. Other sensors demand AI applications to further improve data interpretation, such as vibro-acoustic and metal detectors that are used for non-metallic utility detection and rapid utility probing, respectively.
3. Data/Information Fusion
The previous two sections focused on the use of AI to interpret single-source data. In creating virtual sensors capable of mapping underground utilities based on limited information, one must consider the integration of multi-source data in order to improve the quality of the harnessed information and, thus, the accuracy of maps through a process known as data fusion
[53][54][55][56]. For example, Dutta et al.
[57] proposed to fuse GPR and vibro-acoustic data to combine sensing performance in reading through non-saturated soils (GPR) and saturated soils (vibro-acoustic), and Metje et al.
[1] studied the potential of combining GPR, low-frequency quasi-static electromagnetic fields, and acoustic technologies to improve underground utility mapping accuracy. Generally, while the fusion of data is conducted through user-defined rules and metrics, the iterative process is automated for robustness over extensive knowledge and data space
[58]. This section surveys different AI techniques used in fusing data in order to produce maps of underground utilities.
Fuzzy logic was used by Lanka et al.
[59] as an approximate reasoning method to fuse data from different sources. The proposed model determines the probability of a utility to be located at any Cartesian point based on (1) proximity to manholes; (2) perpendicular distance from the point of interest to the nearest known utility line; (3) expert knowledge on a possible underground utility route; and (4) GPR data analysis. Another technique studied by Chen and Cohn
[60] is based on Bayesian data fusion (BDF) to automatically produce refined utility maps by fusing GPR data, surveyed manholes, assumptions on pipe linearity, and utility records. Manholes are investigated to determine the depths and orientations of the underground pipes. This adopted the joint compatibility branch and bound algorithm developed by Neira and Tardos
[61] to establish an initial map by matching utilities discovered from the manhole survey and GPR scan. Prior knowledge used in the BDF is deduced by matching connections in the as-built maps and GPR-detected points. With the Gaussian distribution assumed, the posterior probability of the BDF is estimated from prior and likelihood probabilities. The posterior map is automatically refined through an iterative process guided by a validation threshold.
Recent work by Zhou et al.
[62] proposed a probabilistic pipeline mapping model that fuses records, pipe information from manhole covers, and remote sensing technologies such as GPR and electro-magnetic. The model maximizes the likelihood of fitting a pipeline from the set of detected points. By integrating the Bayesian information criterion, the proposed model avoids overfitting when classifying and fitting detected points into the most probable pipeline. The authors compared the performance of their algorithm with that proposed in Chen and Cohn
[60] and demonstrated fewer mapping errors.
The reviewed literatures combined different data sources, mostly GPR, utility records, manhole survey, and other sensors. One underlying common limitation to most work is the reliance on manhole surveys. Firstly, it restricts the applicability to manhole-related utilities, and secondly, it is burdensome to open up and determine the geospatial coordinates of every manhole in the area of interest at the same time. It should also be noted that techniques relying on as-built maps as prior knowledge can be challenging to apply when such information is unavailable or outdated.