Fifth-generation (5G) and beyond communications are mainly characterized by (i) massive connectivity, (ii) ultra-reliability and low latency, and (iii) increased throughput. Satisfying these objectives in conjunction with the rapid growth of the Internet of Things (IoT) applications represents a challenging task, especially in highly dynamic and heterogeneous environments. A promising approach is to adopt unmanned aerial vehicles (UAVs) as aerial user equipments (UEs) or flying base stations (BSs). In particular, UAV-based communications can improve the network performance in emergency situations by providing rapid service recovery and by offloading in extremely crowded scenarios. These characteristics have attracted the interest of the standardization bodies [1] and academia. Furthermore, the integration of artificial intelligence (AI) and machine-learning (ML) techniques in wireless networks can leverage intelligence for addressing various issues. Thus, the combination of AI/ML and UAVs appears to be strongly correlated in different disciplines and applications and throughout the network layers, promising unprecedented performance gains and complexity reduction. In the following subsections, a short introduction on the areas of UAVs and ML is presented while relevant surveys are discussed, identifying the current gap in the literature that has motivated the current work.

UAV Characteristics

As the demand for comprehensive broadband services, global coverage, and ubiquitous access has grown, non-terrestrial networks (NTNs) can strongly support the well-established terrestrial backhaul networks [2]. Among the principle components of NTNs, the low-altitude platforms (LAPs) intend to facilitate various civilian, commercial and governmental missions, as well as IoT applications [3], ranging from military and security operations to entertainment and telecommunications. UAVs, the major representative type of LAPs, is usually small unmanned aircrafts employed for short time periods (up to several hours) allowing the rapid deployment of a multi-hop communication backbone in challenging applications without any personnel involved, such as public safety, search and rescue missions, surveillance inspection, emergency communications in post-disaster situations or unexpected events, photographic reconnaissance, urban traffic surveillance, precision agriculture, and media traffic monitoring [4,5,6]. It is worth noting that the market research forecasts that the sales of UAVs will exceed $12 billion per year by 2021 [7], whereas the Federal Aviation Administration (FAA) predicts that the UAVs will count about 2.4 million units by 2022 [8]. It is obvious that such market size and dynamics have been the driving force towards the evolution of UAVs. Recently, great interest has risen by standardization bodies, industries, and academia towards the use of UAVs as flying BSs, mobile relays, or autonomous communicating nodes for providing low latency and highly reliable communications in cities, across suburban areas, and over rural terrains. The use of aerial vehicles for Long-Term Evolution (LTE) was suggested by the 3rd Generation Partnership Project (3GPP) [1], whereas the notion of nomadic relay was presented by the IEEE 802.16s Relay Task Group [9]. In 2013, realistic UAV frameworks have been also established by the special committee (SC-228) formed by the Radio Technical Commission for Aeronautics (RTCA) in an effort to frame performance specifications for the operation of UAVs [10]. Moreover, in 2016, RTCA constituted a drone advisory committee to safely introduce the UAVs into the national airspace system [11]. The National Aeronautics and Space Administration (NASA) and FAA have also launched a joint research initiative to integrate UAVs in national airspace system across the United States [12]. From an industry perspective, major vendors, i.e., Google, Facebook, Microsoft, Qualcomm, Nokia, and Huawei, have also tested the operability of aerial platforms for current LTE and future fifth-generation (5G) applications [13,14,15,16,17,18].

Depending on their flying mechanisms, UAVs can be classified into remotely piloted vehicles (RPVs), multi-rotor drones (also known as rotary-wings drones), fixed-wing drones, hybrid fixed/rotary wing drones, robot planes, and pilotless aircrafts and can differ in size from small toys to large military aircrafts [4,5,6]. Also, the payloads of UAVs including communication equipment, cameras, radars, and sensors vary from tens of grams up to hundreds of kilograms and directly determine their size, the battery capacity, and the flight duration. Owing to their unique characteristics, the UAVs are capable of providing ubiquitous and cost-effective wireless access over large coverage areas at high elevation angles and modest altitudes and with high chance of line-of-sight (LoS) connection with the ground nodes, while they ensure rapid deployment and movement on demand [4,5,6]. Apart from using a small number of UAVs, a swarm of UAVs can also cooperatively work to carry out complex tasks in significantly large areas [19] and especially in monitoring and surveillance applications, whereas the flying ad hoc networks (FANETs) [20], in which multiple UAVs communicate in an ad-hoc manner, can effectively expand the connectivity and communication range in scenarios with terrestrial network constraints, i.e., remote nodes, highly mobile nodes, and highly dispersed nodes. Nevertheless, challenges regarding the mobility, resource management, and control of the UAVs are imposed, especially due to large UAV swarms and the variability of their types, while the successful and long-term operation of UAV-based networks necessitates effective interference mitigation along with coordination and interoperability between heterogeneous wireless systems. Moreover, the limited endurance of the UAVs with respect not only to the networking and on-board processing tasks but also to the power demand of their engines and the flight control is currently one of the major practical factors restricting the full-scale deployment of UAVs in NTNs. In this respect, drone lifetime prolongation is a major concern, strongly related to flight characteristics and mission parameters.

Nevertheless, compared to terrestrial wireless networks, UAV networks have many distinctive features, such as highly dynamic network topologies, orbits or flight paths, and weakly connected communication nodes. Since the power supply is limited, energy-efficient design of airborne systems with respect to path planning and battery scheduling is also required to extend flight duration. Moreover, the mobility and the respective Doppler shift is increased and the Quality of Service (QoS) in data transmissions may be asymmetric. Overall, the communication requirements need to be adapted to the rate and quality of the data transmission in order to achieve the desired performance metrics. Additionally, existing conventional communications techniques encompass inherent limitations, particularly in the case of complex communication scenarios, where unexpected and nonlinear phenomena prevail. As dynamic and mission-critical UAV-based communications lead to particular complexity, uncertainty, and high degree of variability, AI/ML is the key technology that enables fundamentally different decision-making capabilities to obtain a proper UAV placement and trajectory.

Artificial Intelligence and Machine Learning

AI has been considered the science of training machines in order to perform human tasks. There are many applications that AI has been involved in, including robotic vehicles, speech recognition, machine translation, and recently wireless communications. Moreover, a specific subset of the AI is the techniques that are used for training machines in how to learn, which originates a new framework known as ML. In this context, ML can provide solutions in scenarios where a massive number of devices simultaneously requires access to the network’s resources in a dynamic, heterogeneous, and unpredictable way, e.g., in IoT communications. In this sense, intelligent management should be performed in the entire network in order to cope with the various demanding requirements of this novel type of services. The scope is to adaptively and in real-time manage the network resources in an optimal manner. Therefore, ML algorithms have been proposed as an efficient approach for confronting all these contradictory challenges coming from the IoT ecosystem.

In general, ML is based on the pattern recognition framework and its main idea is to exploit correlation among a set of data and/or past good action sequences for adapting to the environmental changes without any kind of human intervention. Clearly, the advantage offered by the ML framework in the wireless networks operation is that it will enable network elements to monitor, learn, and predict various communication-related parameters, such as wireless channel behavior, traffic patterns, user context, and devices locations. ML is classified into various categories, such as supervised learning, semi-supervised learning, unsupervised learning, and reinforcement learning (RL) [21].

• Supervised learning: In supervised learning, the algorithms use data sets, in which both the input and the desired output are available. Therefore, this kind of algorithm can only be employed in scenarios where enough labeled data are available to be exploited.
• Unsupervised learning: The unsupervised learning algorithms also require data to be available for training, which, however, do not include labeled output. Therefore, in this type of learning, clustering or pattern discovery is performed on the available data.
• Semi-supervised learning: An intermediate approach regarding the nature of the available data has been followed with the semi-supervised learning algorithm. In this type of learning, both labeled and unlabeled data are exploited for the training.
• Reinforcement learning: In RL, the problems are solved by employing a sequence of actions that use the trial and error rule. Therefore, the main idea of this type of learning is radically different as compared to the previous mentioned ones, which exploit historical data. Instead, RL algorithms are trained by the previous taken decisions towards solving the problem. The RL algorithms are used in various scenarios in the area of wireless network optimization.

In addition, a specific class of ML is deep learning (DL). In DL, multiple layers have been employed in order to build an artificial neural network, which is able to make intelligent decisions without any kind of human intervention. DL algorithms can be applied when limited manual interference is necessitated, with the cost of higher computational requirements. Nevertheless, AI, ML, and DL methods have been widely employed in various wireless communications scenarios for improving a numerous parameters of the network.

Previous Survey Works

Along with the growing number of novel solutions for wireless networks during the last years, several recent surveys focusing on the interplay of AI/ML and wireless communications have been provided. Various studies have proposed the application of ML for improving the performance of wireless networks. The authors in Reference [22] provide a brief description on the use of DL in an architecture based on the aerostack framework, which is an aerial robotics architecture consistent with the common components related to perception, guidance, navigation, and control of unmanned rotorcraft systems, with different robotic agents consisting of different abstraction levels of unmanned aerial robotic systems, like social, reflective, and deliberative layer, among others. The DL algorithms are employed in three directions: feature extraction, planning and situational awareness, and motion control. The classification is limited to the learning type of the algorithm and the field of application for aerial robot systems which use specific sensors. Next, the survey in Reference [23] studied the use of deep reinforcement learning (DRL) for several network and communication aspects, such as network access and rate control, cashing and offloading, security and connectivity preservation, routing, resources, and data collection. In some cases, the UAV is considered as an agent for the cellular network, like a base station, a sensor, a mobile, a user, a network controller, or a scheduler. Deep Q-network (DQN), deep Q-learning (DQL), and liquid state machine (LSM) are some of the learning algorithms used for the aforementioned issues for the UAV scenarios. A rather comprehensive survey from the exclusive angle of DL applications in mobile and wireless network is depicted in Reference [24]. The authors provide classifications for different attributes, such as mobile big data, data and mobility analysis, user localization, network control, and security. However, the use of UAV networks is not taken into consideration. At the same time, several of the DL techniques that are discussed can be of importance to UAV networks. Then, the survey in Reference [25] provides a brief overview of ML techniques for wireless big data analytics. One of the described applications is related to UAVs, still without discussing AI/ML aspects. Another work discusses, among others, that the AI/ML algorithms on edge devices in 5G and beyond 5G networks are necessary for the development of 6G networks [26]. The critical applications of self-sustaining networks (SSNs) in 6G requires low-latency, high-reliability, and scalable AI, along with a reliable infrastructure, relying on the integration of UAVs and ground network nodes.

In networks consisting of ground-based and aerial vehicles, there have been various surveys presenting possible applications of AI/ML. First, the exploitation of AI for vehicle-to-everything (V2X) application is given in Reference [27], including comparisons of the AI algorithms. Moreover, in autonomous driving, the survey categorizes the most widely used AI techniques in swarm intelligence, ML, DL, expert systems and planning, scheduling, and optimization, where one may find similarities to the prominent UAV classification. Some of the suggested open source or proprietary software tools can also be used in UAV communications, but most of them consider applications like safety, network congestion, navigation, security, content delivery, and edge computing. Furthermore, the paper in Reference [28] gives an overview of 5G communication aspects for V2X, UAVs, and healthcare use-cases. More specifically, UAV-based wireless networks aspects are mostly discussed for disaster management purposes without considering ML enhancements, while ML is mainly considered for healthcare applications in the area of anomaly detection for patients. Next, the authors in Reference [4] present the integration of UAV networks for 5G and beyond communications. Several newly founded techniques are considered (e.g., non-orthogonal multiple access (NOMA) and millimeter wave band (mmWave) transmissions), while a classification of different techniques is performed in terms of physical layer; network layer; and joint communication, computing, and caching, taking into consideration the BS type and the number of UAVs. However, only a limited number of works on the application of ML in UAV networks is included. Then, the survey in Reference [6] offers a thorough overview on using UAVs for cellular communications in view of experimental and simulation channel studies that have been performed in such networks, along with prototyping. In the latter, only two platforms are presented making use of ML algorithms for trajectory and placement. The rest of the paper is dedicated to standardization, regulation, and security aspects, while ML is only included as a possible future research direction for UAV networks.

From the cyber-security point of view, one may find a limited number of survey papers, studying AI/ML. The article in Reference [29] provides a well-structured review of UAV networks from a cyber-physical system (CPS) perspective. The three CPS components in the UAV networks that the survey analyzes are communication, computation, and control. Regarding AI/ML techniques, the authors classify open-source software projects and libraries, mainly focusing on the machine vision area. The authors briefly mention the use of ML and RL at the physical layer and routing, correspondingly, at the computation and communication components. Regarding the computation and control components, intelligent algorithms are discussed for computation-enhanced flight and formation control. The survey in Reference [30] does not focus on communication issues, while the authors provide a mapping of UAV network connectivity, QoS, security, design challenges, and requirements for cyber-physical (CP) security applications. The application of ML is presented for object detection and image recognition, as well as for interesting future research areas, such as collision avoidance, spectrum sensing, channel estimation, and energy management. The main goal of Reference [31] is to outline the wireless and security challenges that arise in the context of UAV-based delivery systems, real-time multimedia streaming, and intelligent transportation systems. To address such challenges, artificial neural network (ANN)-based solution schemes are introduced. The authors consider the security at higher communication layers. Regarding the wireless challenges and relevant AI/ML solutions, each use case follows a different approach and ANN-based solution. Also, the authors provide a discussion on physical layer communication issues, such as interference management. Finally, the work in Reference [5] provides a comprehensive study on the use of UAVs in wireless communications and networks from the CP security perspective. Two main use cases of UAVs are investigated: aerial wireless BSs for 5G and beyond applications that complement emerging wireless communication systems and cellular-connected users that use existing wireless infrastructure. For each use case, key challenges, applications, and fundamental open problems are noted, along with the mathematical tools and techniques needed for addressing the challenges. Regarding the latter, the tutorial provides only a small section discussing the emerging field of ML, primarily for trajectory and path planning.

The majority of surveys either focus on ML for different wireless network environments or foresee the integration in UAV-based networks, discussing its future potential. Nonetheless, three recent works can been identified as more similar to this study. First, the survey in Reference [32] focuses on AI for robotic applications, comprising both ground-based nodes and UAVs. In such scenarios, UAVs can improve the connectivity and security of multi-robot teams towards efficiently performing their tasks. AI/ML techniques for aerospace robotics include (i) ANN for delay, connectivity and security improvement, as well as channel prediction; (ii) particle swarm optimization for determining UAV trajectory; (iii) DL for improved UAV connectivity; and iv) ML for user content request prediction from the UAVs. Then, the article in Reference [33] focuses on a multi-layered communication system comprising space (satellite), air (UAV), and ground segments, namely space-air-ground integrated network (SAGIN). It is found that DL, mainly convolutional neural networks (CNNs), with different DL architectures and training methods can improve the network performance. Next, the tutorial in Reference [34] provides a classification of ANNs for wireless communications. Among several applications (e.g., caching, multiple radio access technologies (RATs), and IoT), the authors refer to UAVs and discuss RL for coverage, connectivity, trajectory, resource, and path planning optimization. A cellular use case for applying AI is considered, where UAVs are part of cache-enabled BSs. Finally, recurrent neural network (RNN) and especially echo state network (ESN) algorithms at the conceptor are studied for mobility prediction.

Departing from these works, in this survey, we focus on the application of a broad set of AI/ML techniques for UAV-based communications without focusing on particular AI/ML categories or UAV applications. More specifically, the following contributions are given:

• An exhaustive overview of AI/ML solutions from all possible categories and their application in UAV-based networks is presented.
• A wide range of UAV-enhanced wireless communication issues, ranging from physical layer and resource management aspects up to trajectory design and caching is studied, while wireless security and public safety applications are comprehensively discussed.
• Open issues are identified for both networking and security areas, stimulating further research for the application of AI/ML techniques in UAVs-based networks.

Table 1 provides brief descriptions of the relevant surveys and the research areas and scope in the context of AI/ML that they discuss.

Table 1. Relevant surveys and tutorials on artificial intelligence (AI)/machine learning (ML) and unmanned aerial vehicle (UAV) communications.

Also, in Figure 1, various applications of the AI/ML solutions in UAV-enabled communications are depicted. Generally, the importance of distilling intelligence in wireless communication networks has been outlined in numerous works. In Reference [24], the authors have observed that the ever increasing heterogeneity and complexity of mobile networks has made monitoring and management of network elements intractable. Moreover, ML allows systematic mining of valuable information from mobile data and automatically identifies correlations that are too complex to be derived by human experts. Likewise, the work in Reference [33] has noted that, in wireless networks, ML enables the wireless devices to actively and intelligently monitor their environment, exploiting mobile data for training purposes in order to learn, predict, and adapt to the evolution of environmental features, including wireless channel dynamics, traffic and mobility patterns, as well as network composition, among others. In this way, they can proactively act towards maximizing the probability of satisfying different performance metrics. It can be seen that ML takes as input data from different sources and through the application of various learning techniques, i.e., supervised/unsupervised, deep or reinforcement learning, allows the network to adapt to the wireless environment in a dynamic and autonomous manner. Thus, especially in networks consisting of a large number of nodes, such as those consisting of swarms of UAVs [35], centralized coordination and excessive overheads that must be acquired and exchanged among the network nodes are avoided, paving the way for distributed network optimization where intelligence plays a key role.

Figure 1. Applications of the AI/ML in UAV-based communications.

Moreover, Figure 2 provides a classification of UAV-based communication network aspects, where the adoption of ML techniques can enhance their performance compared to conventional optimization techniques. Stemming from this taxonomy, the rest of the paper is organized as follows.

Figure 2. Classification of the AI/ML solutions in UAV-based communications.

The publication can be found here: https://www.mdpi.com/1424-8220/19/23/5170/htm