UAV Detection and Tracking in Urban Environments: Comparison
Please note this is a comparison between Version 2 by Xiaochen Yan and Version 1 by Xiaochen Yan.

无人机Unmanned aerial vehicles (UAVs) have gained significant popularity across various domains, but their proliferation also raises concerns about security, public safety, and privacy. Consequently, the detection and tracking of UAVs have become crucial. Among the (UAV) 在各个领域都广受欢迎,但它们的扩散也引发了对安全、公共安全和隐私的担忧。因此,无人机的检测和跟踪变得至关重要。在无人机监测技术中,适用于城市物联网(IoT)环境的技术主要包括射频(RF)、声学和视觉技术。UAV-monitoring technologies, those suitable for urban Internet-of-Things (IoT) environments primarily include radio frequency (RF), acoustic, and visual technologies. 

  • anti-UAV
  • UAV detection
  • UAV tracking
  • UAV dataset
  • anti-UAV systems

1. 简介Introduction

随着技术的快速发展,无人机(With the rapid advancement of technology, unmanned aerial vehicles (UAV)在救援行动s) have found numerous urban IoT (Internet of Things) applications in fields such as rescue operations [1],监视,surveillance[2,3],边缘计算,edge computing[4,5],灾区重建,disaster area reconstruction [6,7],空中基站, aerial base stations [8,9],智能交通, intelligent transportation [10,11],无线电力传输, wireless power transfer [12]、环境监测, environmental monitoring [13], and more [14,15]。根据市场预测,无人机市场的复合年增长率(. According to market forecasts, the UAV market will experience a compound annual growth rate (CAGR)为7.9%,从26年的2亿美元扩大到2022年的38亿美元) of 7.9%, expanding from USD 26.2 billion in 2022 to USD 38.3 billion in 2027 [316]。然而,无人机的使用越来越多,也引发了对安全、公共安全和隐私的担忧。为了解决这些问题,研究人员一直在探索各种无人机监测技术。无人机监视可以通过四种主要技术来实现:雷达、射频. However, the increasing use of UAVs also raises concerns about security, public safety, and privacy. To address these issues, researchers have been exploring various UAV monitoring technologies. UAV surveillance can be achieved through four primary technologies: radar, radio frequency (RF)、视觉和声学监视。(RF), visual, and acoustic surveillance.
雷达监视是一种成熟且广泛采用的空域监视和侦察方法。该技术可用于检测缺乏无线电通信或完全自主的无人机。多普勒雷达是一种强大的技术,它利用多普勒效应来检测移动物体的速度。微多普勒雷达是多普勒雷达的高级版本,可以检测物体内部的速度和运动差异。微多普勒雷达是探测无人机的理想方法,因为它们的螺旋桨会产生较大的线速度差 Radar surveillance is a well-established and widely adopted method for airspace monitoring and reconnaissance. This technology can be used to detect UAVs that lack radio communication or with full autonomy. Doppler radar is a powerful technology that uses the Doppler effect to detect the velocity of moving objects. Micro-Doppler radar is an advanced version of Doppler radar that can detect speed and motion differences inside objects. Micro-Doppler radar is an ideal method for detecting UAVs because their propellers create large linear velocity differences [17]。其他可能适用的雷达技术包括非视距雷达、超宽带雷达、毫米波雷达. Other radar technologies that may be applicable include non-line-of-sight radar, ultra-wideband radar, millimeter-wave radar [18]、混沌单静态、双静态, chaotic mono-static, bi-static [19]和多静态雷达, and multi-static radars [20]。由于其雷达横截面低. Because of their low radar cross section [21]和速度慢,用雷达探测无人机既困难又复杂 and slow speeds, detecting UAVs with radar is difficult and complex [22]。实验结果表明,雷达的探测范围很少超过. Experimental results indicate that the detection range of radar rarely surpasses 10,000 m [23]
可以拦截和分析发射射频波信号的无人机,以跟踪和定位它们. UAVs emitting RF wave signals can be intercepted and analyzed to track and locate them [24]。在许多情况下,手动操作的无人机与地面站和. In many cases, manually operated UAVs communicate with a ground station and a GNSS (GNSS(全球导航卫星系统)通信进行操作,从而可以拦截信号并获得坐标和视频馈送等信息。然而,仅依靠机载传感器进行操作的自主无人机可能不会发出射频信号,这使得定位和跟踪它们更具挑战性lobal Navigation Satellite System) for operation, making it possible to intercept signals and obtain information such as coordinates and video feeds. However, autonomous UAVs that rely solely on onboard sensors for operation may not emit RF signals, making it more challenging to locate and track them [17]。射频监控表现出在. RF surveillance exhibit the capability to detect and locate UAVs within a range of 5000 m 范围内检测和定位无人机的能力。但是,它们的性能可能会受到多径和非视距传播等因素的影响。
近年来,基于计算机视觉的方法在检测和监控无人机方面越来越受欢迎。通过利用深度学习技术,可以训练模型自动从无人机图像和视频数据集中提取外观和运动特征,从而能够使用摄像头的视频监控来识别和跟踪这些车辆. However, their performance can be influenced by factors such as multipath and non-line-of-sight propagation. In recent years, computer vision-based methods have become increasingly popular for detecting and monitoring UAVs. By leveraging deep learning techniques, models can be trained to automatically extract appearance and motion features from datasets of UAV images and videos, enabling the identification and tracking of these vehicles using video surveillance from cameras [25]。此外,红外摄像机可用于在低环境光条件下检测和识别无人机. In addition, infrared cameras can be used to detect and identify UAVs in low ambient light conditions [26]。通过将计算机视觉技术与红外摄像机相结合,可以在各种照明和环境条件下有效地检测和监控无人机。基于视觉的方法在区分无人机和鸟类方面遇到了挑战,特别是当无人机位于相当远的距离时。事实上,识别. By combining computer vision techniques with infrared cameras, UAVs can be detected and monitored effectively in various lighting and environmental conditions. Vision-based methods encounter challenges in distinguishing UAVs from birds, particularly when the UAVs are situated at a considerable distance. In fact, identifying UAVs beyond a range of 1000 m 范围以外的无人机变得极其艰巨,如果不是几乎不可能的话。
在无人机导航过程中,动力装置和螺旋桨叶片发出的声波可以使用声学麦克风进行检测,该麦克风将这些声波的压力转换为电信号。这些声波为每架无人机创建个性化的becomes exceedingly arduous, if not nearly impossible. Sound waves emitted by the power unit and propeller blades during UAV navigation can be detected using an acoustic microphone, which converts the pressure of these sound waves into electrical signals. These sound waves create an individualized 音频指纹audio fingerprint,从而实现个人识别。然而,由于环境噪声和声波衰减等因素,检测这些声波在实践中可能具有挑战性。深度学习模型已被应用于提高该方法在无人机检测中的有效性。通过使用声学特征数据集训练这些模型,他们可以学习区分无人机信号与噪声和其他干扰,从而提高这种方法的准确性和可靠性 for each UAV, enabling individual identification. However, detecting these sound waves can be challenging in practice due to factors such as ambient noise and sound wave attenuation. Deep learning models have been applied to improve the effectiveness of this method for UAV detection. By training these models with datasets of acoustic signatures, they can learn to distinguish UAV signals from noise and other interference, enhancing the accuracy and reliability of this approach [27]。然而,声学监测极易受到环境噪声的影响,并且检测范围有限。根据进行的测试,无人机的最大探测范围不超过. Nevertheless, acoustic monitoring is highly susceptible to ambient noise and possesses a constrained detection range. As per the conducted tests, the maximum detection range for UAVs does not exceed 300 m
. Table 1提供了上述监测技术的全面概述。重要的是要承认,所提供的检测距离来自现有的文献和系统,并且可能会根据无人机类型、硬件参数和相关算法等因素而表现出变化。为了提高无人机识别的有效性,无人机检测系统通常利用上述两种到三种技术的组合 provides a comprehensive overview of the aforementioned monitoring techniques. It is crucial to acknowledge that the detection distances presented are derived from existing literature and systems, and may exhibit variations based on factors such as UAV type, hardware parameters, and associated algorithms. To improve the effectiveness of UAV recognition, UAV detection systems typically utilize a combination of two to three of the aforementioned technologies [22].
Table 1.无人机监视技术的比较。 Comparison of UAV surveillance technologies.
过去,无人机监控系统主要部署在机场和军事基地等关键的军事和民用设施中。然而,随着无人机的日益普及,对无人机监控系统的需求已扩展到更广泛的环境,包括建筑工地、社区、购物中心、学校和其他地点。这创造了对更具成本效益、可扩展性和响应能力更强的无人机监控系统的需求。为了应对这一挑战,研究人员一直在探索使用低成本和无源传感器检测无人机的方法In the past, UAV-monitoring systems were primarily deployed in critical military and civilian facilities such as airports and military bases. However, with the increasing popularity of UAVs, the need for UAV-monitoring systems has expanded to a wider range of settings, including construction sites, communities, shopping malls, schools, and other locations. This has created a demand for UAV-monitoring systems that are more cost-effective, scalable, and responsive. To meet this challenge, researchers have been exploring ways to detect UAVs using lower-cost and passive sensors [28]。软件定义无线电的发展大大降低了射频检测的成本,使其更容易被更广泛的用户使用。近年来,神经网络增强的射频检测、基于视觉的检测和基于声学的检测已成为城市环境中通用无人机监测系统的有前景的选择。随着轻量级模型的发展,已经有一些模型可以用很少的计算资源获得可接受的结果,这使得使用边缘计算进行无人机检测成为可能。虽然雷达监视在探测飞机方面非常有效,但由于与该技术相关的高成本和辐射,其使用仅限于特定位置。因此,它可能不适合在城市地区检测非法无人机。. The development of software-defined radio has greatly reduced the cost of RF detection, making it more accessible to a broader range of users. In recent years, neural network-enhanced RF-based detection, visual-based detection, and acoustic-based detection have emerged as promising options for general UAV-monitoring systems in urban environments. With the development of lightweight models, there are already some models that can obtain acceptable results with very little computing resources, which makes it possible to use edge computing for UAV detection. While radar surveillance is highly effective in detecting aircraft, its use is limited to specific locations due to the high cost and radiation associated with the technology. As a result, it may not be suitable for detecting illegal UAVs in urban areas.

2. 无人机检测与识别 UAV Detection and Identification

在本节中,我们将对基于射频、基于声学和基于视觉的方法进行全面的分析和比较,用于检测和识别城市物联网环境中的无人机。我们将从各种角度探讨这些方法,包括传统方法、深度学习方法以及可用的公共和半公共数据集。此外,我们将讨论与识别物联网环境中的无人机入侵相关的具体挑战,并回顾最近专注于利用边缘设备进行无人机检测的研究。In this section, we will conduct a comprehensive analysis and comparison of RF-based, acoustic-based, and vision-based methods for the detection and identification of UAVs in urban IoT environments. We will explore these methods from various perspectives, including traditional methods, deep learning methods, and the available public and semi-public datasets. Additionally, we will discuss the specific challenges associated with identifying UAV intrusions in IoT environments and review recent research focused on leveraging edge devices for UAV detection. Table 2提供了各种无人机检测方法的性能比较,以及从文献研究中获得的数据。基于深度学习的视觉方法即使在未专门针对无人机优化的公共数据集上进行实验时也表现出良好的准确性。然而,最近的文献研究表明,在专门的无人机数据集上进行微调、优化和训练后,准确率可以超过 provides a performance comparison of various UAV detection methods, with the data obtained from literature research. The vision method based on deep learning demonstrates good accuracy even when experiments are conducted on public datasets that are not specifically optimized for UAVs. However, recent studies in the literature have shown that after fine-tuning, optimization, and training on specialized UAV datasets, the accuracy rate can exceed 90%,表明具有很高的竞争力。, indicating a high level of competitiveness.
Table 2.不同无人机检测方法的比较。 Comparison of different UAV detection methods.

类别

Category

方法

Method

准确性

Accuracy

m

传统的基于射频的检测Traditional RF-based Detection

低雷达横截面

Low radar cross section

分形维数Fractal dimension (FD) [29]

100%

基于延迟的本地化Delay-based localization

轴向积分双光谱axially integrated bispectra (AIB) AIB)

低速和低海拔Low speed and altitude

[30]

98%

Acoustic

平方积分双光谱square integrated bispectra (SIB) SIB) [31

基于 TDOA\AOA 的本地化-based localization

]

0–300 m

高环境噪音High ambient noise

96%

视觉Vision

基于运动的跟踪Motion-based tracking

100–1000m

与鸟类混淆Confuse with birds

信号频谱Signal Spectrum(SFS) [32]

97.85%

不可分割的小物件Indistingushable small objects

小波能量熵Wavelet energy entropy (WEE)[32]

93.75%

射频RF

Power spectral entropy (PSE) [32]

基于 RSS\AOA 的本地化-based localization

83.85%

RF-based Detection Using Deep Learning

Y. Mo et al. [33]

99%

C. J. Swinney et al. [34]

100%

S. Lu et al. [35]

98%

Z. He et al. [36]

90.2%

T. Li et al. [37]

98.57%

Traditional Acoustic-based Detection

Mel Frequency Cepstrum Coefficient (MFCC)

97%

Acoustic-based Detection Using Deep Learning

S. Al-Emadi et al. [27]

96.38%

Q. Dong et al. [38]

99%

İ Aydın et al. [39]

98.78%

Vision-based Detection Using Deep Learning

RCNN

58.50%

SPPNet

59.20%

Fast RCNN

70.00%

Faster RCNN

73.20%

YOLOv3

63.40%

SSD

76.80%

RetinaNet

59.10%

Traditional RF methods involving RF feature extraction have made progress in recent years. The RF approach based on deep learning uses technology to convert RF signals into images, facilitating feature extraction using deep networks. Traditional acoustic methods mainly rely on Mel-Frequency cepstrum coefficient (MFCC) feature extraction, often supplemented by linear predictive cepstrum coefficient (LPCC). Acoustic methods based on deep learning convert sound signals into spectrograms and extract relevant features. In recent years, vision-based detection methods, especially those based on deep learning, have gained prominence. The models listed in the study use public datasets that are not specifically optimized for UAV detection. However, when these models are optimized and trained on specialized UAV datasets, detection accuracy can often exceed 90%. The effectiveness of UAV detection methods depends on factors such as dataset quality, training techniques, and optimization for specific use cases.

3. UAV Localization and Tracking

UAV localization and tracking play a crucial role in anti-UAV research. This section provides a review of UAV localization and UAV tracking methods. UAV localization research primarily concentrates on utilizing RF and acoustic-based methods. These methods are preferred due to their ability to accurately determine the location of UAVs. Regarding UAV tracking, we categorize the studies into filter-based approaches and deep Siamese networks approaches. Filter-based methods utilize various filters for tracking UAVs, while deep Siamese networks approaches leverage deep learning techniques for tracking. The advancements made in these research areas are thoroughly discussed, highlighting the progress and innovations achieved in UAV tracking.

3.1. UAV Localization

RF-based positioning technology has become increasingly mature, leading to numerous studies on RF-based UAV positioning. RF sensors are the only technology that can locate both the UAV and the pilot. On the other hand, acoustic-based UAV positioning technology is feasible for short-distance detection [22]. Although acoustic-based UAV localization technology started later than other methods, it has achieved remarkable results in recent years. In contrast, vision-based UAV tracking technology has its advantages, but due to the rapid movement of UAVs, the effect of visual positioning technology is not ideal.

3.2. UAV Tracking

UAV tracking is a type of object tracking task that involves estimating the state and trajectory of the UAV. Object tracking (OT) is a fundamental open problem that has been extensively studied in the past decade. Two prominent paradigms for OT are Filters-based methods and deep Siamese Networks (SNs) [102].

4. Anti-UAV System

In recent years, there has been a significant amount of literature on anti-UAV systems, and several anti-UAV systems have been used commercially and militarily. One such system is DedroneTracker [113], a multi-sensor platform (RF, PTZ camera, radar) with countermeasure capabilities (jamming) released by Dedrone. The system can be extended and customized to meet specific field requirements and can automatically capture a portfolio of forensic data, including UAV make, model, time and length of UAV activity, and video verification. Dedrone also offers RF sensors and jammers, with a detection range of up to 1.5 km (up to 5 km in special cases) and a maximum jamming range of about 2 km. Droneshield [114] is another system that provides anti-UAV defense solutions, offering a range of standalone portable products as well as rapidly deployable fixed-site solutions. It employs a variety of surveillance technologies, including radar, audio, video, and radio frequency, to detect UAVs and provide effective countermeasures. Droneshield’s jammer can immediately cease video transmission back to the UAS operator and allow the UAV to respond to a live vertical controlled landing or return to the operator controller or starting point. Another RF-based UAV detection system is ARDRONIS [115], developed by Rohde and Schwarz. ARDRONIS performs detection for frequency-spread spectrum (FHSS) signals and WLAN signals, with a detection range of up to 7 km for commercial off-the-shelf remote signals and up to 5 km for UAVs such as the DJI Phantom 4 under ideal conditions. The AUDS [116] system, on the other hand, uses radar technology, video, and thermal imaging to detect and track UAVs and has directional radio frequency inhibition capabilities. With its Air Security Radar, the range of detection can be up to 10 km. The ORELIA Drone-Detector system [117] consists of an acoustic sensor and software for protected object monitoring, target tracing, and sensor adjustment. The detection range is about 100 m, and multiple acoustic detectors can be installed to detect UAVs in all directions. Falcon Shield [118] is a rapidly deployable, scalable, and modular system that combines electro-optics, electronic surveillance, and radar sensors. ELTA Systems [119] is another solution that combines radar, RF, and photoelectric sensors to detect and track UAVs more than 5 km away and to take soft and hard kill measures against them. Based on our observation, the more mature anti-UAV systems are currently mainly focused on military applications, and these systems are often implemented using a combination of multiple sensors. RF monitoring, radar, and vision sensors are the three most commonly used types of multi-sensors. However, radar is not suitable for use in certain locations, such as urban environments, due to its strong radiation characteristics [22]. In urban environments, RF surveillance emerges as a crucial method for locating UAV pilots, while visual surveillance plays a pivotal role in capturing UAV flight videos as evidence of intrusions. This combination of RF and visual surveillance techniques provides a highly competitive solution for UAV detection in urban settings. Although the acoustic-based method has shortcomings such as small detection range and being easily affected by noise, it can also provide an effective supplement for UAV detection under certain conditions, such as poor visual conditions and complex electromagnetic environments. Inspired by the above anti-UAV systems, we propose several suggestions for developing effective and scalable general anti-UAV systems. First, it is essential to consider the specific needs and requirements of the application scenario when selecting and combining sensors. Indeed, various environmental factors can impact the effectiveness of different surveillance methods in urban environments. For instance, visual surveillance may face challenges in locations with low visibility caused by heavy fog, sand, or dust. Similarly, complex electromagnetic environments can negatively affect the performance of RF surveillance. Additionally, acoustic surveillance may not be suitable in areas with strong winds or high levels of ambient noise. It is essential to consider these factors when selecting the most appropriate surveillance method for UAV detection, ensuring optimal performance in diverse urban scenarios. Second, the detection and tracking components of the system should be integrated with appropriate countermeasure capabilities. One of the most prevalent countermeasure equipment against intruding UAVs is the use of RF jamming guns. These devices are designed to disrupt the communication and control signals of the UAVs, forcing them to either land or return to their point of origin. RF jamming guns are widely employed as a common UAV countermeasure. Another commonly utilized countermeasure involves deploying UAVs to capture intruding UAVs. This approach involves using specially designed UAVs equipped with nets, cables, or other mechanisms to physically intercept and capture the unauthorized UAVs. Both RF jamming guns and UAV capture methods serve as effective countermeasures in mitigating the risks and potential threats posed by unauthorized UAV activities. Third, the system should be designed to be scalable and modular, allowing for easy deployment and adaptation to changing conditions. The design concept put forth by Dedrone Company offers valuable inspiration. It emphasizes the importance of creating scalable anti-UAV systems with a platform at the core. Such a design enables easier integration and maintenance of sensors in the future. By adopting a scalable approach, anti-UAV systems can adapt to evolving threats and technological advancements, ensuring flexibility and efficiency in the long run. Fourth, data analytics and machine learning techniques can be employed to enhance the accuracy and efficiency of UAV detection and tracking. Indeed, machine learning techniques have already found extensive application in the field of UAV detection. The continuous advancements in machine learning algorithms and the availability of more comprehensive datasets are key factors in enhancing UAV detection capabilities. By leveraging more efficient learning methods and utilizing diverse and representative datasets, the accuracy and effectiveness of UAV detection systems can be significantly improved. This ongoing development in machine learning holds promise for further advancements in UAV detection technology. Fifth, although accuracy is an important evaluation indicator, lightweight anti-UAV systems that sacrifice part of the accuracy seem to be more in demand in some non-important scenarios. In mobile scenarios or situations where budget constraints exist, lightweight anti-UAV systems that can operate on portable devices offer a more suitable solution. These systems, designed to be lightweight and portable, can be easily deployed and utilized in various environments. They provide flexibility and cost-effectiveness, making them a practical choice for scenarios where mobility and budget considerations are important factors. By leveraging lightweight anti-UAV systems, organizations can enhance their capabilities for UAV detection and mitigation while maintaining operational efficiency. Furthermore, compatibility with existing sensors, such as ubiquitous video surveillance equipment, could significantly reduce the cost of anti-UAV systems. This suggestion of leveraging existing video surveillance equipment, inspired by Dedrone’s products, is indeed valuable. By ensuring compatibility and utilization of the already deployed video surveillance infrastructure, the deployment costs of anti-UAV systems can be significantly reduced. This approach aligns with the concept of using edge computing to assist in UAV detection, as previously discussed. By calibrating the physical locations of these monitoring devices, it becomes possible to detect and track illegally intruding UAVs effectively. This application scenario showcases the potential for cost-effective and efficient UAV detection by leveraging existing resources and edge computing capabilities.
Video Production Service