2.2.2. Communication and Deployment
2.2.3. Overcoming GNSS Limitations
UAV-to-UAV communication plays a critical role in improving the efficiency and adaptability of UAV swarms. It enables the coordination of tasks, the transmission of status information, sharing of sensor data, and collaborative localization among UAVs. UAV swarms can also act as a network of base stations to provide communication services in areas with poor or no wireless network infrastructure. These UAV-swarm base stations can enhance the performance of the network by providing flexibility and adaptability.Global Navigation Satellite System (GNSS) is a technology that provides global positioning and navigation services through satellite systems. It is essential to robot navigation, but the localization to UAV might be restricted under GNSS denial environments. For the discovery of complex circumstances, such as cave environments, the paper [14] developed an autonomous system utilizing the flying robot for cave environment perception. The system is composed of multiple subsystems to achieve full autonomy of UAVs. The requirements of the UAV are also proposed including operative under lacking sources of light and access to GNSS.
2.2.4. Marine and Offshore Operations
The UAV system could also play an important role in marine and offshore operations, such as assessing fire disasters on offshore oil platforms or rescuing drowning victims [15]. In [16], a system is developed for searching castaways by multi-UAVs based on artificial neuronal networks. The system predicts the location range of victims based on the wind and sea currents while keeping track of the discovered victim and finding the rest at the same time. It lays little focus on the UAV charging problem and more information could be considered as clues for searching for victims.
2.2.5. Energy Efficiency during Operation
The energy aspect is essential for the SAR operation of UAVs. Drones are limited by their battery level [17], which means, in other words, the power supply is inadequate for SAR missions. As the former context illustrates, the factors that affect the energy consumption of a UAV are various, including its maximum speed, operating altitude, devices equipped, and its weight (with payload). To study the energy supply, Ref. [18] compares different properties of different battery types (including Pb-acid, NiMH, Li-ion, NiCad, etc.) and points out that lithium polymer batteries and lithium-ion batteries are the most common battery types for UAV, which is because of their high energy and power per unit of battery mass. The UAV could also utilize solar energy for charging, which converts light into electric current through the photovoltaic (PV) effect. The relevant research on solar-powered UAVs has been developed recently, including their path planning [19] and target tracking operations [20].
2.2.6. Artificial Intelligence (AI) integration
Nowadays, AI brings higher autonomy and adaptability to drones, enabling them to play greater roles in various applications. The applications of AI involve many aspects, including a control system by AI-based throttle and elevator control [21], autonomous UAV navigation [22], and learning-based reactive obstacle avoidance maneuver control frameworks [23]. Multiple applications of UAV communication to different AI algorithms are also discussed in the paper [24]. The proposed integration provides an efficient system and enhanced quality of service. The following sections will introduce more AI-integrated techniques based on the specific operations.
2.2.3. Overcoming GNSS Limitations
Global Navigation Satellite System (GNSS) is a technology that provides global positioning and navigation services through satellite systems. It is essential to robot navigation, but the localization to UAV might be restricted under GNSS denial environments. For the discovery of complex circumstances, such as cave environments, the paper [34] developed an autonomous system utilizing the flying robot for cave environment perception. The system is composed of multiple subsystems to achieve full autonomy of UAVs. The requirements of the UAV are also proposed including operative under lacking sources of light and access to GNSS.
2.2.4. Marine and Offshore Operations
The UAV system could also play an important role in marine and offshore operations, such as assessing fire disasters on offshore oil platforms or rescuing drowning victims [7]. In [8], a system is developed for searching castaways by multi-UAVs based on artificial neuronal networks. The system predicts the location range of victims based on the wind and sea currents while keeping track of the discovered victim and finding the rest at the same time. It lays little focus on the UAV charging problem and more information could be considered as clues for searching for victims.
2.2.5. Energy Efficiency during Operation
The energy aspect is essential for the SAR operation of UAVs. Drones are limited by their battery level [40], which means, in other words, the power supply is inadequate for SAR missions. As the former context illustrates, the factors that affect the energy consumption of a UAV are various, including its maximum speed, operating altitude, devices equipped, and its weight (with payload). To study the energy supply, Ref. [41] compares different properties of different battery types (including Pb-acid, NiMH, Li-ion, NiCad, etc.) and points out that lithium polymer batteries and lithium-ion batteries are the most common battery types for UAV, which is because of their high energy and power per unit of battery mass. The UAV could also utilize solar energy for charging, which converts light into electric current through the photovoltaic (PV) effect. The relevant research on solar-powered UAVs has been developed recently, including their path planning [42] and target tracking operations [43].
2.2.6. Artificial Intelligence (AI) integration
Nowadays, AI brings higher autonomy and adaptability to drones, enabling them to play greater roles in various applications. The applications of AI involve many aspects, including a control system by AI-based throttle and elevator control [48], autonomous UAV navigation [49], and learning-based reactive obstacle avoidance maneuver control frameworks [50]. Multiple applications of UAV communication to different AI algorithms are also discussed in the paper [51]. The proposed integration provides an efficient system and enhanced quality of service. The following sections will introduce more AI-integrated techniques based on the specific operations.
3. Application of UAVs in SAR
3.1. On-Site Monitoring, Modeling, and Analysis
Among multiple applications, UAVs are utilized as a tool for environment perception, providing video streaming and 3D models of disaster areas for the planning of SAR operations. During catastrophes such as earthquakes and floods, urban areas are extremely dangerous since the breakdown of buildings might be triggered by the aftershock, soaking, or the impact of high-speed floating obstacles, endangering the safety of SAR teams. The amendment information could be obtained by UAV to prevent the rescue team’s injuries and guarantee the rescue operations’ efficiency.
3.1.1. Monitoring and Modeling on Disaster Area
At disaster sites, traditional methods of manual mapping and measurement may be limited in terms of time and accuracy due to the varying degrees and extent of the damage. However, UAVs can use their efficient data collection and processing capabilities to conduct 3D modeling after the disaster, generating accurate digital maps, 3D models, and volume measurements to support rescue efforts.
Early in 2011, the paper
[53][25] introduced a 3D modeling solution, a low-cost UAV system along with its components, device, and software, to build the 3D point cloud from the digital image. To generate the 3D model based on the image or video stream provided by the UAV camera, the concept of Structure from Motion (SfM)
[54][26] is introduced in the paper
[55][27] as a low-cost solution that compensates the camera motion and estimating the 3D model of the object. Based on the SfM, multiple software packages are developed for constructing 3D models based on the point cloud, such as PhotoScan, Pix4D Mapper, MicMac, and MeshLab.
3.1.2. Building Quality Estimation on Disaster Area
The collapse of buildings during rescue contributes to significant difficulties for SAR operations. Therefore, the applications of UAVs in disaster areas also include building inspection, which is used to analyze the possibility of collapse and plan the rescue operation. For example, the paper
[60][28] developed an image collection system with the post-processing program, which integrates canny mask into edge detecting technique for defect detection. Basic defect-detection operations are achieved in this project.
There also exist multiple defect-detection methods operating on buildings and bridges using UAV photographs, including bridge inspection by videos
[61][29], building inspection by post-processing photographs
[62][30], and defect detection using computer vision
[63][31]. The result of the UAV bridge fatigue detection in
[64][32] even showed that the best performance is comparable to human inspectors.
Although the result shows the good feasibility of the proposed method, the limitations encountered are various
[65][33], including high requirements on the camera, the influence of vehicle motion on the image quality, and unsatisfying GPS signal. The paper
[62][30] found unusable data accounting for a large portion of data acquired, while the realignment and stitching of images are also difficult, which also verified the finding from
[64][32] that camera specification and illumination requirements are essential to the detection performance. Therefore, to overcome these limitations mentioned above, an optimized path pattern and camera setting are proposed in
[66][34], for a UAV system operating 3D modeling using the SfM method, and analyzing the damage by inspecting the small cracks on the example building, which is located 8km north of Napa.
Since computer vision and neural networks can provide functions such as feature extraction and image processing, object detection, and recognition, it is extremely useful for drones surveying building conditions. The paper
[67][35] further introduced speeded-up robust features (SURF)-based feature detection algorithm to achieve the image stitching, while the result showed that image stitching is achieved and the feasibility of UAV inspection on structural damage is verified. Besides the computer vision, The paper
[68][36] utilized a convolutional neural network to distinguish the collapsed and damaged buildings in seismic disaster areas. A 3D model of the building is formed using GIS information at first, and segmentation processing is utilized to separate each building for detailed analysis with multi-view images. CNN-based damage assessments are operated based on the information provided. The experiment in old Beichuan town showed the prediction accuracy reaches 89.39%.
3.2. Perception and Localization of Targets
Perception and localization of targets are always the most important section when conducting search and rescue operations. Thanks to the high-speed mobility and position advantage, UAVs can be used to quickly survey large areas and identify potential victims, allowing rescue teams to target areas of need quickly and accurately. In the former cases illustrated, UAVs could identify hard-to-reach areas and provide real-time updates on the location and condition of victims. The paper
[73][37] reported two cases using UAVs achieving the SAR operation, including victim searching in Dry Creek Canyon, Oregon, by a DJI Phantom 4K quadrotor, and Wahclella Falls Trailhead, Oregon, by an SAR Bot, which is equipped with a thermal imager and made by Aerial Technology International. In both missions, the victims were found deceased in an area that the SAR team had difficulty reaching, which also secured the SAR teams and improved efficiencies.
Multiple works of literature focusing on utilizing the advantage of computer vision are introduced, to reduce the involvement of human operators and cut down human factors. Computer vision refers to the field of artificial intelligence that involves enabling computers to interpret and understand visual data from the sensor installed, such as images or videos. The relevant techniques include the integration of histograms of oriented gradients (HOG) detector to camera
[74][38]. The literature presents the testing result of the HOG detector with other different detectors including the poselet detector, the discriminatively trained part-based models (DPM) detector, and pictorial structures (PS) with the discriminant part detector.
Convolutional Neural Networks (CNNs) are another technique that is commonly used in computer vision tasks, which particularly are designed to automatically learn and extract features from input images, through a series of convolutional layers, pooling layers, and fully connected layers as
Figure 5 describes. In
[6][39], a trained support vector machine (SVM) classifier and Convolutional Neural Networks (CNN) image representation is used for discovering the victim. Pre-processing procedure in the algorithm finds the area of interest by using a sliding window to identify the color differences in HSV color space. The post-processing procedure improves the decision by hidden Markov models (HMM)
[75][40]. The proposed detection obtains better accuracy than the traditional HOG classifier.
Figure 5. Illustration of CNN.
In traditional search and rescue operations, rescuers visually scan large areas of interest to locate missing persons or objects. It allows SAR operators to use their human expertise and judgment to identify and locate the target of interest. Among the literature focusing on human–machine interactive SAR operations, multiple sensors, and videos processing techniques are adopted for the SAR operation to improve the searching performance of human operators, including the RGB data selection
[81][41], temporary localized mosaic view
[82][42], thermal imagery
[83][43].
However, when visual scanning is operated by human rescuers, it can be time-consuming and difficult, particularly in remote or mountainous areas. Strategies proposed to mitigate such human factor also includes the inherited limitation from the method itself. The paper
[81][41] uses the commercial package Loc8 to find the high-contrast color, but the low-contrast color may restrict the performance of the Loc8.
Thermal images provide high-contrast targets for human rescuers, but environmental interference (i.e., object occlusion and temperature increase) and equipment quality will affect the performance of the thermal image. The existing limitations may induce human errors in detecting victims in operation and a decline in efficiency.
The utilization of RGB and thermal cameras are also of great importance to the development of autonomous UAV search and rescue operations.
Thermal camera cues provide high-contrast images between the target and the environment, enabling the target to be explicitly distinguished. However, the intrinsic limitation of thermal cameras requires other sensors to be fused to ensure the robustness of SAR operation. To study the feasibility of thermal cues, the paper
[85][44] integrates four passive infrared detectors (PIR) sensors and two ultrasonic sensors into the UAV system for the requirements of victim detection and collision avoidance. The communication module installed on the system transmits the GPS output to the base station when PIR sensors detected the targets. To enhance the system searching ability, the paper
[4][45] fused an infrared camera with an avalanche beacon receiver to discover both snow-covered and uncovered bodies in the avalanche rescue operation.
3.3. Search and Rescue Operation
3.3.1. Task Assignment
Control of UAVs in SAR operations can be carried out manually by expert pilots, but one of the problems is that coordination among pilots is difficult because of the dynamic nature of the environment during a disaster. So the challenging problem of automatic search by multiple drones has received much attention, and it is a non-deterministic polynomial (NP) problem of combinatorial optimization under multiple constraints. The successful and efficient allocation of available resources will be a solver of such a situation, in which rescue efficiency can be maximized. Vehicles should be able to quickly, reliably, and efficiently find answers to the question: considering the resources available in the network and the tasks that should be performed, what is the best allocation of these tasks among us? The key to solving this problem is to establish the task assignment model and use the assignment algorithm.
The current reality is that most of the environments are uncertain, dynamic, and only partially observable, so it is difficult to implement a centralized global optimization algorithm. Individual agents need to independently and possibly myopically decide what to do next based on the information they receive. Various distributed methods have been proposed. One of the most basic algorithms is the opportunistic task allocation Strategy (OTA)
[97][46], in which an unmanned aircraft randomly selects blocks in an unexplored search area. One way to do this is when information is extremely scarce. There are also some complicated distributed methods, such as market-based methods and consensus algorithms. The key to the market-based approach can be visualized as the auctioneer announcing a bidding task, each agent sending the bid to the auctioneer, and the highest bidder robot winning the task. In a multi-drone SAR operation, the bid value is calculated based on the distance between the drone and the survivor so that the survivor can be rescued in the shortest possible time. However, this method requires a connected topology and a large amount of data transmission to send bids
[98,99][47][48].
3.3.2. Path Planning
The path planning problem for a UAV may be viewed as an optimization problem
[105][49] in which the most common goal is to find a feasible path from the beginning location to the terminal position while following different optimization parameters and constraints. The SAR mission is not always in the open wilderness and may sometimes be in a cluttered and obstacle-rich environment; for example in an urban area or indoor environment, it is necessary for a UAV to adopt a path planning algorithm ensuring the traversed path to be collision-free and optimal in terms of path length. According to the actual situation, there will be some variations. For example, to realize the comprehensive use of UAVs and mobile charging stations, VRP with the synchronous network (VRPSN) is defined in
[106][50], which is a new kind of VRP.
Many methods for UAV path planning have been proposed in recent years. The most common ones are sampling-based methods, such as RRT and PRobability roadmap; there are graph-based methods for designing paths, such as Voronoi graph algorithm, concluded Dijkstra algorithm, A* algorithm, and Markov decision processes
[107,108][51][52]. It is worth mentioning that
[108][52] integrates target motion prediction with the tracking trajectory planning (as
Figure 6 shows), enabling the advanced path planning utility. Applications in this area have been studied as early as 2011; the paper
[109][53] preliminarily addresses SAR using quadrotors. The paper
[110][54] used a hill-climbing algorithm, iteratively, to first find a path at each step by the hill-climbing algorithm, and then optimized the objective function to assign the search effort (flight time) to each cell in the path. The paper
[111][55] broke two assumptions and extended a framework for probabilistic search based on decision-making to merge multiple observations of grid cells and changes in UAV altitude, enabling small, light, low-speed, and agile UAVs, such as quadrotors, to perform occupancy network-based search tasks.
3.3.3. Collision Avoidance
It is not difficult to find from the above that path planning is to generate a set of path points that bypass obstacles from the initial position to the final goal, while collision avoidance takes a given waypoint assignment as a local goal to avoid obstacles.
Early collision avoidance algorithms mainly targeted static obstacles. Since path planning also needs to be considered in obstacle avoidance, the predominant idea for obstacle avoidance in the 1970s and 1980s is to construct a configuration space, and many improved path-planning algorithms have been proposed. However, none of these classical algorithms can minimize the input energy and achieve the optimum results while avoiding obstacles. In the middle of the 1980s, some path-planning algorithms considering uncertain and dynamic environments were proposed, such as potential functions
[113][56], control theory, and other heuristic algorithms. These algorithms solve the shortcomings of classical algorithms but still face challenges when dealing with complex moving obstacles.
After the 2000s, with the development of new technology and improved hardware computing power, more and more obstacle avoidance algorithms have been proposed, making UAVs more agile and robust. The paper
[152][57] describes the safety evaluation process that the international community has deemed necessary to certify such systems about UAVs. The paper
[153][58] proposes an adaptive tracking controller based on output feedback linearization that compensates for dynamic changes in the quadrotor’s center of gravity. The paper
[154][59] combines the improved Lyapunov Guidance Vector Field (LGVF), the Interfered Fluid Dynamical System (IFDS), and the strategy of varying receding-horizon optimization from Model Predictive Control (MPC) to track the target and avoid obstacles in a complex dynamic environment.
4. Summary
In conclusion, the use of unmanned aerial vehicles for search and rescue operations is a promising development that has improved the efficiency and effectiveness of SAR missions. The benefits of drones, including ease of deployment, low maintenance cost, high mobility, and sensor integration, make them an attractive option for SAR operations. The continued development of drones and their sensors will optimistically lead to further advancements in the field of SAR, making it possible to save more lives in the event of disasters and emergencies.