1. Introduction
In general, U
nmanned A
erial Vehicles (UAVs) Vs can be categorized by using construction and altitude. Depending on the construction, UAVs are either fixed-wing or rotary-wing. While most commercial UAVs are rotary-wing types, fixed-wing UAVs are used for very high-speed operations and can carry much heavier payloads. On the other hand, rotary-wing UAVs can fly at low speeds and has outstanding mobility. The latter type of UAV became popular for many potential applications. In terms of altitude, UAVs also can be found as high-altitude platforms which are deployed for long-endurance surveillance and can fly at altitudes above 17 km and remain almost stationary. Low-altitude UAVs, on the other hand, are designed to move quickly at an altitude of a couple of meters up to a few kilometers
[1].
Generally, a UAV is a part of an Unmanned Aerial System (UAS). The following subsystems are the main parts of a regular UAS.
The ground control station acts as the central control unit of the overall UAS, where all the data (video, command, and telemetry) received from the UAV is analyzed and monitored for further decision-making. For smaller UAV applications, communication over a range of up to several kilometers often uses a remote-control system. Satellite systems may be involved in extreme UAV operations where the ground station is located thousands of kilometers from the UAV work zone
[2]. Recently, autonomous UAVs or autonomous swarms of UAVs were proposed that may operate in the field without continuous communication with the GCS.
Data-link or COM is a part of UAVs that provides duplex communication with the ground control station and other UAVs. To safely and reliably operate the UAV, a stable communication system is an important requirement. The COM is mainly composed of a transmitter, receiver, antenna, modulator, etc. Recently the fifth-generation (5G) communication, both the 5G base station-based and device-to-device (side-link) protocols, becomes widely used for this purpose.
UAV sensors can be broadly categorized as critical sensors for Inertial Measurement Units (IMUs) and navigation and tracking sensors for route planning and object detection
[3]. The critical sensors such as accelerometer, gyroscope, magnetometer, compass, ultrasound height, and pressure sensors are mainly used for flight control to measure the altitude and rotational axis. Image sensors such as video cameras (monocular or stereo), Light Detection and Ranging (Lidar), Radio Detection and Ranging (Radar), and lasers are employed to capture videos and images in path planning and object (stationary or moving) detection. Motors and associated electronics drive circuits act as actuators in UAVs.
Similar to other embedded systems, a UAV needs a CP as a processing system that retrieves data from payloads and other sub-modules. The processed information is then delivered to the actuator and ground station or another UAV to operate the UAV successfully. Most of the CPs of commercial and civil UAVs are μC- or COTS-based embedded systems. However, modern UAVs are performing complex image processing and real-time object detection with the help of ML, DL, and other types of mathematical algorithms. UAVs deployed for applications implying computation-intensive processing use high-speed multi- and many-core processor systems, Graphical Processing Units (GPUs), FPGAs, All Programmable Systems-on-Chip (APSoCs), SoC-FPGA, to process their complex tasks efficiently.
In the following subsection, rwesearchers will explore CP used for FMs of UAVs in high-level and low-level applications.
2. Flight Control Computer
The Flight Control Computer (FCC) of FMs plays a significant role in keeping the UAVs in a specific position and returning to the base station properly. If the FCC does not function accurately, there may be a chance of an accident or failing the mission. FCC monitors UAV states continuously through various critical and navigation sensors. The FCC can be sorted by low- and high-level flight control operation
[3]. In low-level, basic flight control operations, such as motor control, UAV stability, and processing sensor data are performed. FCC is often engaged in high-level applications such as autonomous navigation, path planning, stereo vision, simultaneous localization, and mapping that make UAVs autonomous. In high-level operations, FCC requires high processing power where an OS is running over HW/SW co-design to implement complex navigation and object detection algorithms. The modern CP such as SoC-FPGA itself can perform both high and low-level operations
rwe
searchers denoted them in this
researchpaper as hybrid-level, although it also requires the help of other computing devices in critical applications.
2.1. Low-Level FCC
SoC-FPGA: Low-level FCC FMs such as IMU core, receiver IP cores for pulse-width modulation signal, and Proportional Integral Derivative (PID) controllers were designed and developed on a single SoC-FPGA-based CP in work
[4]. In work
[5], the authors proposed four techniques in designing the controller of the FCC considering low power, fast response, and less volume for FPGA- or Digital Signal Processor (DSP)-based small UAVs. Research work in
[6], presented a secured operation for FCC FM by using
μC and FPGA combinedly.
μC controls all sensors and generates the signals for controlling the UAV motors. FPGA handles the data encryption and decryption task before sending data to the UAV’s motor and radio systems.
μC-based FCC: Using several low-cost sensors such as an IMUs and a Lidar, the research work
[7] implemented an
μC-based FCC FM for the small rotary-wing UAVs to estimate the position of the UAV and its distance from an obstacle or a landing field. Employing several low-cost sensors such as a 10-DOF micro-electro-mechanical system IMU and a Lidar, research work
[8] applied the
μC-based FCC FM for small rotary-wing UAVs to determine the location of the UAV and its distance from an obstacle or the landing surface.
2.2. High-Level FCC
SoC-FPGA for Algorithm Implementation: The authors in work
[9] presented SW/HW co-design framework for UAV returning by proposing an improved region-based Kanade-Lucas-Tomasi tracking algorithm. They also improved the hardware acceleration architecture by integrating parallelism and improving resource utilization for FCC FM in the SoC-FPGA-based CP. In the study
[10], the authors developed real-time processing systems such as mean subtraction, windowing, finite impulse response filtering, decimation, and spectral estimation via Fast Fourier Transform (FFT). Their implementation results using similar SoC-FPGA CP achieved real-time 3-dimensional detection of local UAV traffic at a range of 1000 m. Similar work is presented by
[11] where additional processing system for frequency modulated continuous wave phased array Radar utilizing SoC-FPGA for autonomous navigation to identify nearby aircraft such as small UAVs up to 350 m and bigger aircraft up to 800 m. On that CP, DSP algorithms were also employed, including parallel FFT, cross-correlation, and beam-forming. In work
[12], the CORDIC, EKF, and PID-Fuzzy algorithms were integrated with the FCC platform to create a real-time Guidance, Navigation, and Contro (GNC) system on an FPGA to read data from IMU sensors. After processing the payload data, FPGA-based CP generates navigation commands as Pulse width Modulation to actuator and servo motors.
μC-based High-level FCC: A decision-making algorithm based on fuzzy logic was demonstrated in
[13] using the Arduino Uno
μC CP for controlling the IMU of autonomous UAVs. They used an IMU algorithm to predict the parameters of inclination, lateral, and bending angles in flight, which allows the UAV to navigate fast and avoid obstacles. Another study
[14] employed an
μC-based CP with an embedded flight map containing flight information and constraints on the cargo carried and the flying mode.
2.3. Hybrid-Level FCC
SoC-FPGA-based Hybrid-level FCC: A model-based HW/SW co-design was proposed in
[15] for implementing both high and low-level FCC FMs, where they represented and compared four possible boards to implement such operations. In noisy environments, such as where it is cloudy or under trees, the GPS signals are so weak that UAV faces difficulty in tracking and localization. To tackle these real-time challenges, the work in
[16] implemented a real-time vision-based navigation system based on the AprilTag algorithm using the SoC-FPGA CP to perform real-time pose estimation, tracking, and localization in GPS-denied environments. In another similar work
[17], the authors presented an approximate adder design focused on error-tolerant size, weight, and power for intensive UAV imaging applications such as 2-dimensional Discrete Cosine Transform, airborne self-localization, and moving object tracking algorithms.
μC-based Hybrid-level FCC: The basic FCC operations such as dynamic modeling, control system design, model-in-the-loop, and hardware-in-the-loop of an unmanned helicopter were implemented using a novel Linux-based flight control system built on Raspberry Pi board in work
[18]. In
[19], the authors focused on implementing an autonomous source-seeking application using Deep Reinforcement Learning on
μC-based CP for nano quadcopters. They tested their proposed method using open-source CrazyFile nano quadcopters and found it to be 70% more efficient in source seeking. Using a similar
μC-based CP and open-source CrazyFile nano quadcopters, the work in
[20] provided an onboard HW/SW autonomous visual navigation system utilizing a CNN-based DL network.
Table 1 shows the list of research works and their implemented FCC FM along with applications. From this table, researchwers can observe that SoC-FPGA platforms are utilized in most of the cases for conducting both high- and low-level operations.
, authors used the Haar cascade classifier algorithm with OpenCV library in their model and reported that they achieved a 98% True Positive rate for 1.5 m height using the Haar cascade classifier algorithm with OpenCV library in the design. Similar work in
[29], the authors used a CNN algorithm for the classification and obtained 100% accuracy with a distance of object 1–4 m in detecting victims of natural disasters.
Table 2 shows the different CPs used in various object detection and tracking applications of UAVs.
], the author proposed a scalable FPGA-based CNN hardware accelerator for embedded systems based on an 8-bit fixed-point approximation of a hardware-friendly CNN model with the OpenCL framework and obtained 1.9× energy efficiency compared to previous work. Similar works
[31][32] described FPGA-based hardware accelerators for implementing depthwise CNN. These research works also achieved better performance than CPU and GPU in object detection. The authors used coarse-grained and fine-grained parallel computing optimization methodologies to improve computational speed and throughput in an FPGA-based CNN accelerator.
A
multi-sensory fusion technique using infrared and visible light based on CNN for UAV surveillance operations was presented in work
[34].
TIn this study, they built an image fusion approach on two widely used HW accelerators: Zedboard (ARM + FPGA) and NVIDIA TX1 (ARM + GPU), and evaluated the performance, finding that FPGA-based platforms outperform GPU-based platforms. An automated navigation system utilizing both IMU sensors and image processing was employed to estimate the UAV location discussed in the work
[35]. They developed a hybrid computing architecture consisting of FPGA, CPU, and
μC for carrying out the implementation and data fusion process. In work
[36], another multi-sensory fusion task was demonstrated in an energy-efficient way using the Spiking NN on the FPGA-based platform. Their proposed hardware implementation achieved an accuracy of 99.7%.
The research in
[37] achieved higher performances using Zynq FPGA over the conventional GPU as an accelerator to implement CNN-based image processing for
real-time object detection scenarios. To address the issue of the Quality of Experience (QoE), the authors developed an FPGA-based architecture called SCYLLA
[38]. SCYLLA offers a novel reconfiguration-based profile generation technique that generates a pool of FPGA design and Deep Neural Network (DNN) model profiles with different QoE performances. They reported that SCYLLA reduces the processing latency by 11.9× and saves 71.5× of the energy consumption compared to the CPU-based solution. Recently, the You-Only-Look-Once (YOLO) method, a fast and accurate DNN architecture, explored new concepts in real-time multi-object recognition. The authors of
[39] investigated the performance of several SoC-FPGA platforms in real-time object detection and recognition on the YOLO network. A Tiny YOLOv2 was designed in
[40] for the real-time object detection for CNN-based implementation using FPGAs where they achieved 3.19× better than the GPU for the performance-power efficiency. Similarly, in
[41], a YOLOv2 NNA was developed on the FPGA platform by designing an accelerator memory access module. Their evaluation proved that the implemented design performs better balance speed and accuracy compared with similar research results.
Table 3 shows several representative FPGA-based CPs used as accelerators in NNA applications for the UAVs.
3. Computation Intensive Payload
In this section, rwesearchers discuss representative CIP FMs of UAVs such as object detection, tracking, image processing, and NNA applications.
3.1. Object Detection, Tracking, and Environment Monitoring
SoC-FPGA-based Detection: In work
[21], an infrared image processing system was implemented using combined computing platforms of FPGA and DSP for image acquisition, tracking, and matching algorithms.
Terrain classification is important for an emergency landing, aerial mapping, decision making, and cooperation between UAVs in autonomous navigation systems. Using three algorithms (Gray-Level Co-Occurrence Matrix, Gray-Level Run Length Matrix, and Flow), the research
[22] provided a complete solution for terrain classification in differentiating among the four terrain types (water, vegetation, asphalt, and sand). Their proposed solution developed on the FPGA achieved a 95.14% success rate in train classification using the OpenCV library. Another challenge of UAVs in the SAR operation is the
moving target detection. The authors of
[23][24] included speed estimates and object segmentation algorithms to identify real-time moving objects using an area-based image registration method in the SoC-FPGA-based CP.
μC-based Detection: In work
[25], a moving target detection system was implemented while considering avoiding obstacles robustly in heterogeneous swarm of UAVs. Employing
μC-based hybrid controllers, they implemented target seeking and obstacle avoidance calculations separately in a distributed UAV swarm architecture. Similar research work in
[26] presented resource-limited platforms using
μC and GPU for AI-based object detection and tracking. A CNN algorithm is incorporated where an object tracking algorithm is tailored based on a Gain-Scheduled PID controller to follow the detected object under variable speed.
Sometimes, UAVs are used in safety operations to monitor the surroundings as
environment monitoring. For instance, authors in work
[27] mounted the toxic gas detection sensor array on the IoT-based UAV architecture to monitor the air quality in the given environment. They used
μC-based controllers to connect the air sensors and to monitor the sensor data. In a rescue operation using a UAV, people or face recognition is another challenging task that requires a real-time complex processing system. A face detection and recognition system utilizing
μC-based CP can identify disastrous people on the ground with high accuracy. In research work
[28]
3.2. Neural Network Accelerator
In this subsection, rwesearchers discuss several representative CPs used as an NNA FM. NNA is, actually, a special processor designed for an artificial NN-based ML workload.
FPGA as an NNA: In
computer vision tasks such as image classification or segmentation, video analysis, and CNN-related DL algorithms are used intensively in many applications. However, the CNN model is challenging to implement in a resource-constrained UAV due to model complexity and costly computing procedures. Many researchers are now employing FPGA-based hardware accelerators to tackle this issue efficiently
[30][31][32][33]. In the research work in
[30
4. Communication Module
SoC-FPGA-based COM: The authors in work
[42], presented an FPGA-based Channel Emulator for Non-Stationary Multiple Input Multiple Output (MIMO) Fading Channels required for UAV
communication system. They developed several COMs such as a delay module, fading generation, an interpolator for a 2 × 2 MIMO channel implemented in a single FPGA CP which achieved a good performance. A data link terminal controlling several UAVs dynamically was implemented effectively on the FPGA CP in
[43] which focused on digital zero-IF signal processing unit design and hardware implementation process. The interleaver module is an important component in the transmitter and receiver module for stable UAV communication. The research work
[44] implemented that module on the FPGA CP using LUT RAM. The authors, in work
[45], designed an agile digital Software Defined Radio (SDR) system in the SoC-FPGA for the UAV target application. The COMs such as Global Navigation Satellite System, GSM, and WiFi were tested and evaluated on that SoC-FPGA-based CP. In another similar work
[46], the authors presented a downlink and uplink high-speed communication in a rapidly changing propagation environment for short-range UAVs. They implemented their proposed design in the SDR system using FPGA and
μC.
In
[47], the authors proposed a security architecture that uses for UAV
reliable communication and evaluated COM on the FPGA CP involving the transmission of bitstreams between the UAV and ground station. Similar reliable communication between the UAV and ground control station was built in research work
[48] using
μC-based CP for a UAV communication system to evaluate the single-carrier Frequency Division Multiplexing (FDM) modulation technique.
An extensive study was performed to investigate the performance enhancement in the UAV-assisted networks for the
5G and beyond
5G wireless communication system
[49]. The UAV-assisted networks for
5G wireless communication systems can be a promising solution to deploy emergency wireless communication networks to restore connectivity in post-disaster areas. A model for
5G communication networks was developed for post-disaster wireless networks considering FPGA as the implementation unit of a reconfigurable intelligent reflecting surface to find an optimal power allocation
[50][51].
Table 4 represents several CPs used for COMs for the UAVs.
5. Layers of the UAV Computing Platform
RWe
searchers can conclude that CP is the heart of the UAV system; it controls all sub-systems of the UAV. Based on the above analysis of the CPs used in different UAV applications,
researcherswe can illustrate a comprehensive layered representation of UAV systems and its subsystems in
Figure 1 assuming a swarm intelligent application. The lower layer of the figure represents the UAV edge node, where, in the IoT terms, edge computing is performed considerably reducing time delay and energy consumption when performing a complex task such as real-time object detection
[52][53]. The middle layer of this figure represents the UAV swarm intelligence at the fog level, where multiple UAV systems (edge nodes) collaboratively perform real-time complex computing tasks that require offloading technology for the edge UAV to reduce the energy consumption, latency, and throughput
[54][55][56]. Efficient communication between multiple UAVs also needs a resource allocation mechanism that can be applied in the UAV networks to maximize the efficiency of the UAV systems
[57][58]. The wireless communication networks of UAVs could also be affected by potential cyber-attacks as mentioned
[59][60]. Finally, the ground station at the cloud level controls the overall UAV systems by receiving and transmitting the signal. The focus of this survey work is on the CP which is the core processing part for the edge computing of the UAV system as shown in the lower part of the figure. The correct operation of a UAV system is strongly intertwined with the CP’s hardware reliability, necessitating the use of a cross-layer fault-tolerant management system and keeping care of all of the subsystems indicated in the figure. The hardware reliability evaluation for UAVs is discussed in the next section.
Figure 1. Basic overview of UAV system.