2. Developments of UAVs in Bridge Inspection in the US
Due to rapid advancements in UAV technology in recent years, in the US, Departments of Transportations (DOTs) have shown increasing interest in the use of UAV for bridge inspections
[4]. In 2008, California DOT (Caltrans)
[5] and the University of California at Davis designed a custom twin-motor, single-duct UAV to be tethered to the ground, making it easier to control and conform to the FAA regulations at the time. The objective of Caltrans was to construct an “Aerobot” to easily access structural components at high altitudes, such as girders
[6]. Caltrans terminated the project as it did not result in a fully deployable aerial vehicle due to its instability in the wind and the unsuccessful performance of an altitude holder sensor.
Wisconsin DOT
[7] used two different UAVs for evaluating damage conditions specific to each of the three roadway bridges, including two steel girder bridges and one steel truss bridge. They learned that quality of the UAV equipment is important for bridge inspections since the results quality is tied to the resolution of the images and the ability to view the bridge elements from proper angles. Idaho Transportation Department
[8] researched the use of UAVs in under-bridge inspections for detecting fatigue cracking. The conclusion of the experiments was that detecting fatigue cracking and other bridge defects by using visual spectrum and thermal image processing is feasible, but requires a careful selection of UAS platforms, on-board avionics, and data collection sensors
[9].
Michigan DOT (MDOT)
[10] has conducted tests of UAVs for bridge inspections since April 2015. Images taken with a UAV were used to detect deficiencies in bridge decking for potholes and wear, and involved the use of RGB cameras and infrared and LiDAR sensors. The studies demonstrated that using UAV increases safety and reduces inspection costs. It was also found that simultaneous use of different sensors can significantly improve the accuracy of collected data. Oregon DOT (ODOT)
[11] conducted a statewide study on UAV applications for bridge inspections. Wind condition was found to be the most important environmental variable in operating UAV close to bridges, while ambient light conditions and camera settings are critical to obtaining high-quality imagery. The ability to articulate a camera in any direction with a zoom lens and employ an onboard camera-assistant spotlight was the most useful technical feature for collecting visual data. The use of a UAV was most effective for initial and routine inspections and less effective for more complex in-depth inspections that require touching, probing, or scraping a bridge.
Minnesota DOT (MnDOT)
[12] performed one of the most comprehensive studies evaluating UAVs’ application and effectiveness in bridge inspections. In the first phase (2015), the research team learned that UAVs capable of pointing cameras upward and operating without GPS have better performance for bridge inspection applications. In the second phase (2017), the research team expanded the demonstration to different structure types and sizes using a SenseFly Albris UAV. They concluded that this vehicle is flexible to control, and its operational capabilities are not diminished by the loss of GPS signals. In the third phase (2018), 39 bridges including a wide range of sizes, types, and locations are inspected using SenseFly Albris and Flyability Elios UAVs (a quadcopter enclosed in a spherical wireframe cage to avoid collision damage/specialized in performing indoor inspections by utilizing computer vision-based navigation). They learned that easy piloting, access to confined areas, and cost-effectiveness are the benefits, whereas short battery life, video interference due to the cage and air flow, and debris are the limitations of this specific type of drone. It was also found that UAV platforms equipped with thermal sensors can effectively detect concrete delamination. The study recommended using UAV for routine inspections where hands-on inspection is not required. Another recommendation relates to the use of UAV equipped with sophisticated collision avoidance systems as well as the use of collision-tolerant platforms which can operate in direct contact with the bridge structure.
UMass research team
[13] developed and tested practical procedures and protocols to guide MassDOT in integrating UAV technologies into bridge inspections. It was determined that selection of the proper types of UAV platforms and sensors are the primary factors that affect the success of UAV integration into the bridge inspections. Kansas DOT (KDOT), in collaboration with the Kansas State University Transportation Center, studied the potential of UAV implementation within KDOT. They found out there is a need to handle large and overwhelming volumes of collected data
[14]. Nebraska DOT (NDOT) conducted a study on UAV applications in bridge inspection program. The NDOT study concluded that except fracture critical bridge inspections, all other types of bridge inspections could incorporate UAV
[15].
North Carolina DOT (NCDOT) partnered with North Carolina State University to evaluate the potential benefits of UAV for transportation applications. The conclusion of the study indicates that the major factors affecting success of UAV missions include weather, sensor capabilities, flight planning, software processing, and ground control point design and placement
[16]. Vermont Agency of Transportation (VTrans) tested UAV capabilities for bridge inspections. It was noted that significant increases in the volume of data collected with the help of UAV may create additional challenges for data storage and processing. It was also found that photogrammetry can successfully replace LiDAR in the generation of accurate 3D models for lower cost
[17]. Most common types of UAVs which state DOTs have used for bridge inspection purposes are shown in
Table 2.
There are numerous ongoing research programs to eliminate current limitations and address the needs in the application of UAVs in bridge inspection. These include handling large volumes of collected data, environmental conditions affecting the quality of images, navigation and flight stability in areas with poor GPS signals and around large steel structures, collision avoidance capabilities and image processing, and advanced computational methods to detect/evaluate deck deficiencies. A few of these studies are summarized in this section.
To analyze the effectiveness of drones as supplemental bridge inspection tools and facilitate bridge inspection, researchers have conducted numerous studies. Junwon Seo et al. (2018)
[21] performed an analysis of the effectiveness of drone-based bridge inspection. They used a DJI Phantom 4 quadcopter to inspect a bridge. The drone was able to identify various damage types, including cracks, spalling, corrosion, and moisture on the bridge. During the conduction of the study, some limitations were identified, including high wind speeds, camera overexposure, low illumination, and flight challenges due to obstacles in an enclosed section (e.g., between closely spaced girders). S. Sankarsrinivasan et al.
[22] proposed a novel complete field mapping protocol using UAVs to enable their real-time health monitoring. This method integrates data captured by a UAV to identify cracks and assess surface degradation using grayscale thresholding. They used a custom-built hexacopter with a payload capacity of 110 g and a 20 min flight time. Yunas Zewdu Ayele (2020)
[23] proposed a methodology for a UAV-based bridge inspection to assess bridge damage using novel technologies. Their methodology for bridge inspection involves collecting data and training a model which enables modifying drone flights to obtain optimum efficiency. The data gathered from the previous stage is built into 3D models to capture the element geometry of every bridge asset to use for navigational and controlling purposes. Chen et al. (2016)
[24] developed a hexacopter with an upward gimbal that can capture upward imagery and accommodate additional attachments such as an ultrasonic sensor, laser scanner, and LiDAR. They learned that position estimation using a laser scanner can only work in the specific bridge environment, and it is still a problem without GPS when the environment is much more complex. To fly in a more complex environment, other localization methods need to be integrated into the system to get a more precise position to control the drone’s flight. D. Roca et al.
[25] used a Mikrokopter Okto XL octocopter for inspecting outdoor building facades. This UAV has a Kinect sensor mounted unit capable of acquiring geometric data in 3D, which can then be reproduced as a 3D model to evaluate potential damages. HekmatiAthar et al. (2020)
[26] introduced a hierarchical multiple-criteria decision making framework for UAV-enabled bridge inspection selection practices. The initiated framework employed a hierarchical method to analyze 32 criteria categorized in flight performance, situational awareness, payload and sensor capabilities and communication quality.
Even with advances in UAV technology, manual piloting remains a challenge, and developing navigation and autonomous flight capabilities are of growing interest. For example, Yu et al. 2019
[27] presented a LiDAR-based approach for autonomous navigation using 2D LiDAR scanning. Bolourian et al. (2020)
[28] proposed an optimized path planning technique for a UAV-based LiDAR scanner that performs bridge inspections. This technique uses genetic algorithms for solving the traveling salesman problem of potential locations of bridge cracks with an objective of minimum flight time and maximum visibility. Jung et al. 2020
[29] addressed a hierarchical graph-based simultaneous localization and mapping (SLAM) method for fully autonomous bridge inspection using an aerial vehicle. They concluded that even with accurate position estimation under a bridge, the risk of UAV collision significantly increases when a wind gust of over 10 m/s suddenly occurs.
The ability to convert images or video data into actionable information automatically and robustly remains challenging. Vision-based approaches, in conjunction with cameras and UAVs, offer the potential for rapid and automated inspection and monitoring for bridge condition assessment
[30]. Sutanu Bhowmick et al.
[31] concluded that UAVs with real-time vision sensing are more efficient in terms of time and resources. They used pixel segmentation to identify whether a particular pixel belonged to a crack or not. Krisada Chaiyasarn et al.
[32] developed a convolutional neural network (CNN) based image crack detection method for inspecting historical structures using feature extraction. The data is captured using a DJI Phantom UAV. Saleem et al. 2020
[33] proposed instant crack damage detection using an image capturing and geo-tagging system with a CNN for automated inspection. The damages extracted by the CNN are instantly transformed into a global bridge damage map, with georeferencing data acquired using image capturing and geo-tagging. To overcome the limitation of visual inspection in terms of post-processing big data to develop a 3D model, a UAV-based real-time autonomous crack detection algorithm was proposed by Morgenthal et al.
[34]. This system includes machine learning-based feature detection of target areas that provides crack information to the team instantly while capturing data.
The application of infrared thermography (IRT) techniques has been demonstrated in several research projects. Omar et al. (2017)
[35] demonstrated the applicability of UAV-mounted thermal cameras for quantitative measurements of delamination in RC bridge decks. Image analysis based on the k-means clustering technique was utilized to segment the mosaic and identify objective thresholds. Mac et al. 2019
[36] considered simultaneously using the results from the handheld IR camera, and the IR camera mounted on a UAV. They found out that there is a strong correlation between the size and detectable depth of delamination. Hiasa et al. 2017
[37] presented a methodology that combines numerical modeling and IRT data to improve the usability and efficiency of data analysis, possibly leading to automated analysis and evaluation. To obtain thresholds for data processing, finite element model simulation was utilized. Washer et al. (2010)
[38] presented results from a study of the effect of solar loading on the detection of embedded targets in a large concrete block. The effect of the depth of the embedded target is discussed, as well as the timing of inspection (relative to sunrise) that resulted in maximum contrast in thermal images. Ellenberg et al. 2016
[39] demonstrated the capability of UAVs equipped with both color and IR cameras to rapidly and effectively detect and estimate the size of regions where subsurface delamination exists. Shen et al. (2020)
[40] suggested that IRT technology could be a complementary method to evaluate the delamination of concrete bridge decks in addition to the existing chain dragging method currently used by the Nebraska Department of Transportation (NDOT). They mentioned some common pitfalls such as dark asphalt smears on the concrete deck, wet deck surface, and excessive dirt covering the deck surface that must be avoided during the survey.
Ongoing research that addresses the limitations of GPS-based UAV navigation relies on computer vision approaches to seek and avoid obstacles and optimize the flight path. Youn et al.
[41] created a real-time surrounding map for the UAV navigation in a GPS denied environment with the integration of an error state Kalman filter and an open-sourced SLAM (simultaneous localization and mapping) algorithm. To achieve flight control and reachability, a nonlinear observer control technique can be implemented to guide the UAV
[42]. This system uses an onboard UAV sensor suite and a visual camera to identify a target with as low as four feature points and compare them with a preprogrammed feature data library. Based on the feature identified, the UAV takes the necessary position and velocity corrections autonomously without a pilot command
[43]. Flightpath optimization is another major aspect of infrastructure inspection and can be achieved using novel optimization techniques, such as particle swarm optimization
[44].
Custom-made UAVs have gained considerable attention to address the current limitations of off-the-shelf vehicles. Whitley et al. (2020)
[45] presented a solution to the current limitations in the implementation of UAVs which are reliance on a skilled operator and/or the requirement for a UAV to operate in a cluttered, GPS-denied environment. They utilized commercial off-the-shelf hardware, including laser rangefinders, optical flow sensors, and live video telemetry. González-deSantos et al. 2020
[46] presented a new payload to perform contact inspection in large structures using UAV. The payload has been designed to be independent of the flight controller. The payload approaches the structure slowly and avoids bounces during the first touch. This sensor measures the thickness of metal sheets when in contact with it. Myeong et al. 2015
[47] demonstrated the use of wall-climbing UAVs that can fly and stick on walls to perform inspections. Kocel et al.
[48] developed a UAV payload with a combination of a long probe and a transducer that contacts the surface and moves along with the UAV for a thorough inspection. They developed a robust flight control system to maintain minimal distance from the inspecting surface, a major requirement for this UAV technology.