Segmentation and Path Planning of Unmanned Ariel Vehicle

Version	Summary	Created by	Modification	Content Size	Created at	Operation
1		Ahmed Barnawi	--	2030	2023-12-15 10:20:36	\|
2	format change	Peter Tang	+ 1 word(s)	2031	2023-12-15 10:26:23	\|

This entry is adapted from the peer-reviewed paper 10.3390/s23167264

Unmanned aerial vehicles (UAVs), sometimes known as “drones”, are unmanned aircraft that can be flown without a pilot on board. Aircraft, ground control stations, and communications systems all fall under the umbrella term unmanned aircraft systems (UAS), which describes the infrastructure necessary for sophisticated drone operations. An autonomous drone is a UAV that can fly missions independently of a human pilot. It can take off, execute its task, and return to base without human assistance. Rather than relying on a human pilot, communications management software handles mission planning and flight control for autonomous drones.

segmentation coverage path planning deep learning

1. Unmanned Aerial Vehicle (UAV)-Based Landmine Detection

Major application areas for drone technology include agriculture, disaster management, remote surveying, network coverage, and product delivery. In addition to the developments in network technology, UAVs play a greater role in maintaining and enhancing network capabilities. The UAVs play a crucial role in topographical surveying by offering a quick, simple, inexpensive, and safe approach with decreased human participation and improved access to challenging terrain. Recent research has envisioned a similar system for landmine detection ^[1]. Various sensing devices can detect the landmine signature, including metal detectors, ground penetration radar (GPR), infrared, multispectral, acoustic, magnetometer, and others ^[2]^[3]. More specialized sensors are now compatible with UAVs, allowing for more precise and thorough airborne surveys as technology advances.

Figure 1 depicts the process of landmine detection where the minefield survey utilizes a UAV equipped with detection sensors. The area is first chosen for the UAV-based investigation and its geographical map serves as the input. The required portion is then retrieved to determine the region of interest. A route or path plan is established to produce the coverage path in the extracted area. UAVs utilize the created course to explore all geographical places and gather sensor-based information. Information collected may subsequently be processed using artificial intelligence (AI) techniques to help locate and identify landmines. Additionally, the minefield layout may be recognized by post-processing. The demining team can use the detection results to inspect the specified locations and remove or destroy the landmines.

Figure 1. UAV-based landmine detection.

The use of UAVs in landmine detection provides several benefits, including a reduction in survey time, improved accessibility to difficult terrain, and risk-free operation. Consequently, several researchers have investigated the possibility of improving the detection procedure by using UAVs. Using low-cost drones, the visual detection system was proposed in ^[4] to detect fully and partially visible landmines. The system provides good efficiency in a low-altitude survey and low flight speed. The UAV equipped with a metal detector was considered in ^[5] to carry out aerial detection in rough terrains. The developed system can identify metals, mines, and bare explosives buried underground. A metal detector requires a certain amount of metal content to generate the landmine signature. It makes the detection of low-metal landmines difficult using metal detectors. Colorado et al. ^[6] integrated a software-defined radio (SDR)-based GPR system with a UAV that was capable of detecting landmines in variable terrains. The flight control system was developed to enable the steady operation of the UAV. The ground-penetrating synthetic aperture radar (GPSAR) was used with UAV in ^[7] to accelerate the demining process. The strong clutter generated at the surface–air interface affected the GPR-based landmine detection. The time-gating and average subtraction technique was used for clutter removal in ^[8], and SAR processing was used to obtain high-resolution images.

A lightweight and low-power GPR was developed in ^[9], based on the stepped frequency continuous wave (SCFW) radar. The developed system efficiently detects metallic and plastic landmines while flying at low altitudes. But the detection system has limited scanning speed and suffers from the moisture content of the soil. In ^[10], an improved SAR-based GPR detection system was used with UAV for landmine and improvised explosive device (IED) detection. The system provides a better signal-to-clutter ratio in GPR images using clutter filtering based on singular value decomposition (SVD).

The UAV equipped with a thermal and multispectral sensor was employed for scatterable landmine detection in ^[11]. It is based on a convolution neural network (CNN)-based identification and localization of landmine signatures in the captured sensor images. The magnetometer-based survey using UAV was presented in ^[12] to detect different types of mines. The study has taken into account a sensor-equipped UAV with a height of 1 m and a survey line spacing of 3 m. It was improved in ^[13] by altering the sensor position to enable low-altitude operation. The low-pass filtering and moving average methods were used to eliminate the magnetic noise and improve detection.

Various sensing technologies can be used with UAVs for the remote survey of minefields, each serving some advantages and disadvantages in different use cases. In most studies, the survey was remotely controlled in a simple environment. It poses a challenge for UAV operation in complex terrains where the optimal utilization of resources and complete coverage are critical. It requires automation of trajectory formulation for UAV-based surveys.

2. Segmentation

Segmentation is the process of breaking down a picture into multiple pieces or regions ^[14]. It entails dividing a picture into smaller, more manageable regions, with each section being represented by a mask or a set of labels. Simply said, it is the process of assigning labels to the image pixels. The pixels that share the same attributes are labeled alike. It helps in the identification of different image components, and instead of processing the complete image, just the relevant parts need to be processed. The segmentation can be categorized as semantic segmentation, instance segmentation, and panoptic segmentation.

In this research, K-means clustering and normalized RGB color space are used to segment the green terrain in the provided image. Using K-means clustering and normalized RGB color space, the results demonstrate that the shared Google Earth photographs can be divided into segments that are roughly 40.50% and 47.01% of the total image pixels, respectively ^[15].

When image segmentation methods are developed, they could enter the medical area and help with disease diagnosis. Using pathology images to expedite clinical diagnosis and automate image analysis with reliability and effectiveness is still highly challenging. For several machine learning methods, including convolutional neural networks, automatic pathological picture segmentation was proposed. Accuracy and processing speed are provided for this segmentation using fully convolutional networks and other deep learning techniques ^[16].

The segmentation process plays a crucial role in enhancing the efficacy of landmine detection procedures. By precisely identifying and isolating areas with a high probability of containing landmines, segmentation lays the groundwork for targeted and efficient UAV-based aerial surveys. Moreover, the segmentation process assists in overcoming the obstacles posed by complex and diverse terrains. It enables the UAV to prioritize survey activities, concentrating on pre-identified regions more likely to contain landmines. This targeted strategy not only increases survey efficiency but also improves operational safety. By incorporating the segmentation process into the landmine detection workflow, the accuracy, efficiency, and overall effectiveness of UAV-based aerial surveys can be significantly improved. By intelligently identifying regions of interest, the UAV can implement survey missions with greater precision, resulting in enhanced landmine detection results and the establishment of safer environments. As part of the investigation, the researchers collected satellite images from Google Earth for the intended use. The objective is to determine the precise target area for UAV aerial surveys. This requires the initial collection of datasets, followed by the precise definition of the intended area of interest. Then, a deep learning model is utilized to effectively recognize and categorize the distinct regions.

3. UAV Path Planning

In recent years, various studies are focused on automating the UAV flight. One such component examined in the literature to automate the UAV-based aerial survey is autonomous route determination, also known as path planning. It covers the estimation of the best route between a source and a destination ^[17]. In some applications, like archaeological surveys, the objective is to cover every location within a region. It requires the determination of the optimal coverage path, denoted as coverage path planning ^[18]. Figure 2 illustrates the execution of an aerial survey, wherein the UAV systematically traverses a predetermined coverage path to collect data, covering the whole target region.

Figure 2. Representation of the UAV-based survey of the target area.

Several recent works have investigated the coverage path planning of UAVs. Surveys involving UAVs encompass operations under a range of conditions, accounting for varying complexities and the number of regions Accordingly, different strategies have been formulated to automate the coverage path determination. The target region having complex geometry often requires partitioning the free space into cells to simplify the survey path, termed cellular decomposition ^[19]. It can be categorized as the exact and approximate cellular decomposition or grid decomposition. In the exact decomposition method, the region is partitioned into smaller sub-areas, the combination of which forms the same region. The path for each sub-region is then determined, and the integration of all the paths generates the coverage path. On the other hand, the approximate decomposition discretizes the area into a set of regular cells at some resolution. The traversal of all the cells in some order generates the coverage path.

The authors of ^[20] focused on the exact cellular decomposition of the region for a UAV-based survey using a greedy strategy. The back-and-forth motion was used to obtain the coverage path for each sub-region. The minimum spanning tree (MST)-based traversal was used to obtain the complete coverage path for the undirected graph of sub-regions. Torres et al. considered the line sweep direction computation for the polygon, and then the coverage path was generated perpendicular to it. In ^[21], the work focused on UAV operations in the presence of wind, and a dynamic programming-based approach was used for polygon decomposition. The selected direction of the coverage path line was perpendicular to the wind direction to consume the minimum flight time.

The approximate cellular decomposition and a gradient-based approach were used in ^[22] to obtain a coverage path with the minimum number of turns. The energy model for UAV was derived in ^[23], which was used to determine the optimal coverage path along with safety mechanisms. The authors of ^[24] examined the coverage path formulation based on exact and grid-based decomposition methods for various CPP widths.

In some applications, the target area to be surveyed can be considerably large, making it difficult to be covered with a single UAV. Also, certain applications, like disaster management, are time-critical and require the survey to be completed as quickly as possible ^[25]. The survey task can be enhanced by employing multiple UAVs leading to cooperative or multi-UAV coverage ^[26]. The region can be decomposed into smaller fragments that are assigned to different UAVs. It requires careful partitioning and sub-region allocation to optimize the complete coverage ^[27]. The hexagonal decomposition of the target region and clustering-based allocation to multiple UAVs was proposed in ^[28]. Similarly, Morse-based decomposition was used in ^[29] where sub-region assignment considered the UAV start and end positions. Choi et al. ^[30] proposed an optimization model with column generation to track energy consumption during an aerial survey. The work focused on minimizing the number of UAVs and energy requirements for aerial surveys.

Researchers have investigated the cooperative utilization of several UAVs to achieve sustained coverage, with a focus on grids that are visited less frequently. Furthermore, specific strategies have been implemented to address emergency situations in particular regions. In some scenarios, multiple UAVs can operate in a certain formation, and the coverage path can be determined accordingly ^[31]. The authors of ^[32] included mixed-integer linear programming (MILP) modeling of the coverage task and proposed a randomized search-based algorithm to optimize the coverage path for heterogeneous UAVs.

In certain applications, UAVs need to survey multiple separated regions instead of a single area. It requires proper planning and sequence determination if a single UAV is used to cover all the regions to determine an efficient coverage path. Xie et al. ^[33] formulated the problem as the integration of the traveling salesperson problem (TSP) and CPP. Grid-based and dynamic programming (DP)-based approaches have been used to determine each region’s entry and exit point and optimize intra and inter-regional coverage paths. The work was extended in ^[34], where a heuristic method was employed. The exact methods have limited scalability, where the time complexity increases exponentially with the problem scale. The heuristic approaches can provide appropriate solutions in an acceptable time and improved scalability.

References

Barnawi, A.; Budhiraja, I.; Kumar, K.; Kumar, N.; Alzahrani, B.; Almansour, A.; Noor, A. A comprehensive review on landmine detection using deep learning techniques in 5G environment: Open issues and challenges. Neural Comput. Appl. 2022, 34, 21657–21676.
Bello, R. Literature review on landmines and detection methods. Front. Sci. 2013, 3, 27–42.
Kasban, H.; Zahran, O.; Elaraby, S.M.; El-Kordy, M.; Abd El-Samie, F.E. A comparative study of landmine detection techniques. Sens. Imaging Int. J. 2010, 11, 89–112.
Castiblanco, C.; Rodriguez, J.; Mondragon, I.; Parra, C.; Colorado, J. Air drones for explosive landmines detection. In Proceedings of the ROBOT2013: First Iberian Robotics Conference, Madrid, Spain, 28–29 November 2013; Springer: Berlin/Heidelberg, Germany, 2014; pp. 107–114.
Fattah, S.A.; Haider, M.Z.; Chowdhury, D.; Sarkar, M.; Chowdhury, R.I.; Islam, M.S.; Karim, R.; Rahi, A.; Shahnaz, C. An aerial landmine detection system with dynamic path and explosion mode identification features. In Proceedings of the 2016 IEEE Global Humanitarian Technology Conference (GHTC), Seattle, WA, USA, 13–16 October 2016; pp. 745–752.
Colorado, J.; Perez, M.; Mondragon, I.; Mendez, D.; Parra, C.; Devia, C.; Martinez-Moritz, J.; Neira, L. An integrated aerial system for landmine detection: SDR-based Ground Penetrating Radar onboard an autonomous drone. Adv. Robot. 2017, 31, 791–808.
Schartel, M.; Burr, R.; Mayer, W.; Docci, N.; Waldschmidt, C. UAV-based ground penetrating synthetic aperture radar. In Proceedings of the 2018 IEEE MTT-S International Conference on Microwaves for Intelligent Mobility (ICMIM), Munich, Germany, 15–17 April 2018; pp. 1–4.
Fernández, M.G.; López, Y.Á.; Arboleya, A.A.; Valdés, B.G.; Vaqueiro, Y.R.; Andrés, F.L.H.; García, A.P. Synthetic aperture radar imaging system for landmine detection using a ground penetrating radar on board a unmanned aerial vehicle. IEEE Access 2018, 6, 45100–45112.
Šipoš, D.; Gleich, D. A lightweight and low-power UAV-borne ground penetrating radar design for landmine detection. Sensors 2020, 20, 2234.
Garcia-Fernandez, M.; López, Y.Á.; Andrés, F.L.H. Airborne multi-channel ground penetrating radar for improvised explosive devices and landmine detection. IEEE Access 2020, 8, 165927–165943.
Baur, J.; Steinberg, G.; Nikulin, A.; Chiu, K.; de Smet, T.S. Applying deep learning to automate UAV-based detection of scatterable landmines. Remote Sens. 2020, 12, 859.
Yoo, L.S.; Lee, J.H.; Ko, S.H.; Jung, S.K.; Lee, S.H.; Lee, Y.K. A drone fitted with a magnetometer detects landmines. IEEE Geosci. Remote. Sens. Lett. 2020, 17, 2035–2039.
Yoo, L.S.; Lee, J.H.; Lee, Y.K.; Jung, S.K.; Choi, Y. Application of a drone magnetometer system to military mine detection in the demilitarized zone. Sensors 2021, 21, 3175.
Sultana, F.; Sufian, A.; Dutta, P. Evolution of image segmentation using deep convolutional neural network: A survey. Knowl.-Based Syst. 2020, 201, 106062.
Rozanda, N.E.; Ismail, M.; Permana, I. Segmentation Google Earth imagery using k-means clustering and normalized RGB color space. In Computational Intelligence in Data Mining; Springer: Berlin/Heidelberg, Germany, 2015; Volume 1, pp. 375–386.
Wang, S.; Yang, D.M.; Rong, R.; Zhan, X.; Xiao, G. Pathology image analysis using segmentation deep learning algorithms. Am. J. Pathol. 2019, 189, 1686–1698.
Aggarwal, S.; Kumar, N. Path planning techniques for unmanned aerial vehicles: A review, solutions, and challenges. Comput. Commun. 2020, 149, 270–299.
Galceran, E.; Carreras, M. A survey on coverage path planning for robotics. Robot. Auton. Syst. 2013, 61, 1258–1276.
Cabreira, T.M.; Brisolara, L.B.; Paulo R, F.J. Survey on coverage path planning with unmanned aerial vehicles. Drones 2019, 3, 4.
Li, Y.; Chen, H.; Er, M.J.; Wang, X. Coverage path planning for UAVs based on enhanced exact cellular decomposition method. Mechatronics 2011, 21, 876–885.
Coombes, M.; Fletcher, T.; Chen, W.H.; Liu, C. Optimal polygon decomposition for UAV survey coverage path planning in wind. Sensors 2018, 18, 2132.
Valente, J.; Sanz, D.; Del Cerro, J.; Barrientos, A.; de Frutos, M.Á. Near-optimal coverage trajectories for image mosaicing using a mini quad-rotor over irregular-shaped fields. Precis. Agric. 2013, 14, 115–132.
Di Franco, C.; Buttazzo, G. Coverage path planning for UAVs photogrammetry with energy and resolution constraints. J. Intell. Robot. Syst. 2016, 83, 445–462.
Pérez-González, A.; Benítez-Montoya, N.; Jaramillo-Duque, Á.; Cano-Quintero, J.B. Coverage path planning with semantic segmentation for UAV in PV plants. Appl. Sci. 2021, 11, 12093.
Nedjati, A.; Izbirak, G.; Vizvari, B.; Arkat, J. Complete coverage path planning for a multi-UAV response system in post-earthquake assessment. Robotics 2016, 5, 26.
Zhang, H.; Xin, B.; Dou, L.h.; Chen, J.; Hirota, K. A review of cooperative path planning of an unmanned aerial vehicle group. Front. Inf. Technol. Electron. Eng. 2020, 21, 1671–1694.
Araujo, J.; Sujit, P.; Sousa, J.B. Multiple UAV area decomposition and coverage. In Proceedings of the 2013 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), Singapore, 16–19 April 2013; pp. 30–37.
Azpúrua, H.; Freitas, G.M.; Macharet, D.G.; Campos, M.F. Multi-robot coverage path planning using hexagonal segmentation for geophysical surveys. Robotica 2018, 36, 1144–1166.
Guastella, D.C.; Cantelli, L.; Giammello, G.; Melita, C.D.; Spatino, G.; Muscato, G. Complete coverage path planning for aerial vehicle flocks deployed in outdoor environments. Comput. Electr. Eng. 2019, 75, 189–201.
Choi, Y.; Choi, Y.; Briceno, S.; Mavris, D. Energy-constrained multi-UAV coverage path planning for an aerial imagery mission using column generation. J. Intell. Robot. Syst. 2020, 97, 125–139.
Cao, Y.; Cheng, X.; Mu, J. Concentrated Coverage Path Planning Algorithm of UAV Formation for Aerial Photography. IEEE Sensors J. 2022, 22, 11098–11111.
Cho, S.W.; Park, H.J.; Lee, H.; Shim, D.H.; Kim, S.Y. Coverage path planning for multiple unmanned aerial vehicles in maritime search and rescue operations. Comput. Ind. Eng. 2021, 161, 107612.
Xie, J.; Carrillo, L.R.G.; Jin, L. An integrated traveling salesman and coverage path planning problem for unmanned aircraft systems. IEEE Control. Syst. Lett. 2018, 3, 67–72.
Xie, J.; Carrillo, L.R.G.; Jin, L. Path planning for UAV to cover multiple separated convex polygonal regions. IEEE Access 2020, 8, 51770–51785.

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.

Upload a video for this entry

Information

Subjects: Computer Science, Artificial Intelligence

Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :

Ahmed Barnawi

Krishan Kumar

Neeraj Kumar

Nisha Thakur

Bander Alzahrani

Amal Almansour

View Times: 162

Update Date: 15 Dec 2023

Table of Contents

Video Upload Options

Confirm

1. Unmanned Aerial Vehicle (UAV)-Based Landmine Detection

2. Segmentation

3. UAV Path Planning

References