Non-Terrestrial Networks (NTNs), including Low Earth Orbit (LEO) satellite constellations, Unmanned Aerial Systems (UASs), and High Altitude Platforms (HAPs), have been identified as promising technologies to provide ubiquitous connectivity [1] in the future generation Internet. For this reason, the Third-Generation Partnership Project (3GPP) [2] has included NTNs among the supporting technologies for the extension of the terrestrial fifth generation (5G) into the sixth-generation (6G) mobile networks. NTNs can be exploited to meet the requirements of emerging technologies, such as ubiquitous artificial intelligence (AI) and the Industrial IoT (IIoT), for application use cases such as remote monitoring, goods delivery, connected autonomous vehicles (CAVs), and high-speed transportation (e.g., trains or aircraft). However, the main challenge in New Radio NTN integration is the communication between the User Equipment (UE) and the satellite, because it requires the Line-of-Sight (LOS). In dense urban scenarios, high-rise buildings or tall infrastructures can severely affect LOS communication due to signal blockage and reflection phenomena. Communication in the LOS between satellites and the UE becomes even more challenging in scenarios where the satellite and the UE are moving relative to each other because, in these scenarios, the LOS probability changes with the satellite elevation angle. Therefore, continuous LOS estimation techniques are paramount for the UE to access the satellite and maintain connectivity.

2. Multipath Traffic Scheduling and Protection

Multipath traffic scheduling means allocating network traffic on multiple paths by selecting the appropriate path(s) for data transmission to meet specific constraints or service requirements. A multipath scheduling technique must take into account both traffic protection and bandwidth preservation. However, most of the scheduling techniques proposed in the literature do not take either into consideration. For example, the conventional round-robin (RR) scheduling strategy sends data sequentially over multiple paths and neglects path conditions. The RR scheduler has been found to perform poorly in the Multipath Transmission Control Protocol (MP-TCP) ^[3][6]. Although the Weighted Round Robin (WRR) scheduler is improved compared with the (RR) scheduler, the weights it assigns to paths are usually static, which makes it impractical under time-varying conditions such as those in NTN networks. Similarly, the deficit round-robin (DRR) and weighted fair queuing (WRQ) schedulers ^[3][6] do not adapt easily to dynamic channel conditions. Path-Aware Networking (PAN) scheduling strategies consider path conditions such as the Round-Trip Time (RTT), Packet Loss Rate (PLR) and bandwidth ^[4][7]. The RTT as used in schedulers such as the round-trip time threshold and the lowest-RTT-first schedulers ^[3][6] enables traffic to arrive before the expiration time ^[5][6][8,9], but the head-of-line blocking can affect connections that differ greatly in latency. When PLR ^[6][7][9,10] is used for scheduling, undelivered and delayed traffic is taken as lost traffic. In contrast, theour proposed learning-based scheduling system can adapt to dynamic link states and make proper path selections and redundancy estimations. As for traffic protection, several schemes have been proposed, such as Forward Error Correction (FEC) and Automatic Repeat reQuest (ARQ). FEC can waste bandwidth because of the fixed redundancy rate that does not take into account the dynamism of the network. ARQ, on the other hand, uses retransmissions to compensate for lost traffic but can cause network congestion ^[8][9][11,12] and adversely affect multimedia quality ^[10][13]. Thus, it is not preferred for real-time traffic, especially in satellite transmission, which is characterized by long delays. In such scenarios, FEC could provide a solution, but it introduces a computational load on constrained devices such as UAVs. TheOur learning-based model, on the other hand, provides traffic protection by using the required redundancy, depending on the dynamic network conditions, while avoiding excessive overhead.

3. RL-Based Traffic Scheduling

Recently, there has been great interest in applying RL in transmission networks. As a result, various RL-based models have been proposed for traffic control and scheduling. In ^[11][14], the authors presented a scheduling framework based on RL for satisfying the bandwidth requirements of Wi-Fi users. In ^[12][15], another scheduler using the RL model was proposed for multipath QUIC in Wi-Fi and cellular transmissions. An AC framework was proposed in ^[13][16] for dynamic single-user and multi-user access to multiple links in wireless networks which avoids collisions by selecting suitable links. An RL framework was proposed in ^[14][17] to improve data rates and mitigate E2E delay in IoT networks. The work in ^[15][18] proposed an AC-based scheduling framework for UAV cellular integrated networks. Inspired by this development, but different from these works, rwesearchers propose a learning-based framework that performs path selection and traffic protection by repeating traffic over multiple links to provide redundancy. Using an AC-based algorithm, theour agent searches for a policy to select suitable satellite links in terms of LOS availability and determine how much redundancy is required to protect traffic against channel losses due to a varying LOS probability, which depends on the changing satellite elevation angle. Redundancy estimation is carried out in such a way as to provide enough protection without wasting bandwidth. Moreover, the AC algorithm used by theour agent is a model-free RL algorithm that does not require knowing in advance the model’s underlying transmission channels.

4. LOS Prediction and Tracking

Various LOS prediction and tracking methods have also been proposed. For example, in ^[16][19], the authors proposed a theoretical model that estimates the probability of cloud-free LOS (CFLOS) for satellite links based on the satellite elevation and the altitude of the ground station. Sun et al. ^[17][20] proposed a method for detecting Non-Line-of-Sight (NLOS) using data from the Global Navigation Satellite System. In ^[18][21], the authors proposed an empirical model to estimate the LOS probability for satellite and HAP communications. In contrast to these physical and empirical methods, rwesearchers propose an RL-based model for non-stationary scenarios in which LEO satellites move continuously, causing their elevation angles and consequently the LOS probability to change.