Although the sky may seem big enough for two flying vehicles to collide, the facts show that midair collisions still occur from time to time and are a major concern for aviation safety authorities. As a preventative measure, pilots are instructed to keep one eye out of the cockpit, scan the sky for potential threats, and be prepared to maneuver to avoid a potential accident [1,2]. However, this “see-and-avoid” rule has several important limitations. First, the pilot cannot look outside all the time, as he also has to check the instruments inside the cockpit, which measure, for example, air speed or engine temperature. The time spent looking inwards and outwards must therefore be perfectly balanced. Pilots who spend most of their time looking at the instruments, and this is especially true of novice pilots, endanger the aircraft by ignoring other elements circulating around them.

The “80–20” rule suggests that pilots should spend no less than 80% of their time looking out and no more than 20% of their time checking instruments. The 80 does not just refer to looking for other traffic, as the pilot also looks for visual cues used for navigation. In any case, even if a pilot or crew member could spend 100% of their time scanning the sky, this would not mean that no threat could escape the human eye. In fact, the fraction of our visual field that allows us to detect anything in the sky is extremely small. Therefore, for practical scanning, pilots are also instructed to follow a pattern, dividing the horizon into regions and taking a moment (1–2 s) to focus before moving on to the next region. Hence, the horizon is divided into nine regions; the pilot’s eye scans one ninth at a time. In other words, at least 89% of the horizon remains unattended at all times. This gives a clear idea of the chances of a threat escaping the human eye, especially when you consider that a light aircraft, such as a 9-meter-wingspan Piper PA-28 Cherokee, approaching head-on at 90 knots on a collision course, 5 seconds before impact, looks no bigger than a sparrow [3]. To make matters worse, the performance of the human eye can be reduced by cloud cover, glare from the sun, fatigue, and many other factors.

With today’s technologies, such as SSR (secondary surveillance radar), transponders, TCAS (traffic collision avoidance system) and, more recently, ADS-B (automatic dependent surveillance-broadcast), one might think that midair collisions should no longer occur. However, they do, because these technologies are not mandatory in all countries, airspaces, or aircraft. They are also fallible, because human factors still cause accidents. The new century has also brought a new player onto the scene. Since 2005, the use of unmanned aerial vehicles (UAVs) or drones in commercial applications has grown exponentially [4,5], increasing the need for safe path planning [6] and obstacle avoidance [7]. New commercial applications are not without risk, potentially causing damage and disrupting other aerial activities [8]. In 2017, a DJI Phantom 4 collided with a US Army UH-60M helicopter near Coney Island, New York. This is only the first documented case of a UAV collision with a manned aircraft, but the number of UAV sightings by pilots has increased dramatically in recent years. Aircraft collision avoidance is therefore a challenging problem due to the stochastic environment and uncertainty about the intentions of other aircraft [9].

For these reasons, and particularly in the case of manned and unmanned light aircraft in uncontrolled airspace and at aerodromes, various safety agencies and pilot associations are encouraging pilots and UAV users to install some form of electronic conspicuity (EC) device on their vehicles to be more aware of nearby aircraft [10,11,12]. An example of such EC technology is FLARM (FLight alARM, https://flarm.com/ accessed on 11 September 2023). The FLARM predicts potential conflicts and alerts the pilot with visual and audible warnings [13]. This device was originally designed for gliders, which are slower than light aircraft. The main limitation of these devices is compatibility, as a FLARM can only display air traffic that uses another matching FLARM. Incompatibility occurs, for example, when the communication solution is different due to the use of different frequencies (the US version of FLARM devices uses a different band than the European one) or different protocols (a FLARM device that has not been updated for a year is not compatible with the latest version of the protocol and will automatically shut down). In addition, some devices are active, i.e., they transmit and share their position with others, while others are passive, i.e., they listen to the transmissions of others but remain invisible to them (e.g., many portable air traffic detectors only listen to the transponders of other aircraft). In this “Tower of Babel” scenario, when communications fail or are absent, pilots should rely not solely on their eyes to detect threats, but on an artificial eye capable of scanning the sky faster, farther, wider, sharper, and more consistently.

The most used solution for preventing accidents is the use of radars. There are many types of radar used in aviation, but the most important is the primary surveillance radar (PSR). PSR detects the position and altitude of an aircraft by measuring the time it takes for radar waves to bounce off an aircraft and return to the radar antenna. These radar systems can detect both transponder-equipped and non-transponder-equipped aircraft. PSR is not perfect and has its limitations. PSR cannot identify the detected obstacle, and the required equipment is expensive [14]. Computer vision solutions have the advantage in this subject because the equipment is relatively cheap, and depending on the implementation, the solution can identify the incoming obstacle. Nevertheless, computer vision effectiveness can be impacted by light conditions. That is the reason why some researchers try to combine both approaches like [15] to overcome each approach’s limitations.

2. Vision-Based Flying Obstacle Detection for Avoiding Midair Collisions

2.1. Computer Vision

2.2. Testing Tools

2.3. Obstacles and Future Work