A plethora of technologies and combinations for improving position accuracy are presented and evaluated in the literature. Technologies are RF-based (cellular, WLAN, ultra-wideband (UWB), Bluetooth), image detection–based (infrared (IR), optical, LiDAR, RADAR cameras) or sound-based (acoustics and ultrasound (US)). Several past studies have performed surveys on the landscape of indoor positioning systems at the time [9,10,11,12,13,14]. Few of these studies propose specific parameters that help with evaluating indoor positioning systems in an attempt to draw general conclusions about a technology. The authors of [8] mention that throughout this field of research IPS are evaluated in particular environments resulting in conclusions that cannot be generalized, nor compared to other systems’ results. Although various metrics are proposed in the literature for parameterizing environmental characteristics, there are variations in the real world which are difficult to capture. The authors of [12] argue that when choosing between technologies, the parameters should match the application requirements, otherwise a trade-off is designed. The works in [9,11] suggest a taxonomy of features for evaluating systems. Different parameters proposed by [9,11] are used in this study to evaluate the two systems. The parameters used for comparison are accuracy, coverage, scalability, cost and type of environment.

The most popular wireless signals used in device ranging are sound-based and radio frequency (RF). Within sound-based systems, there has been an increase in ultrasound positioning technologies starting with systems such as the one presented in [15]. In the RF domain, positioning systems based on Wi-Fi and Bluetooth flood the literature—recently popularizing Bluetooth Low-Energy (BLE) and UWB [8,16]. RF-based technologies gained traction due to their adaptability to different use cases [7]. With RF signals travelling at the speed of light, penetration characteristics give them an advantage over US signals that require Line of Sight (LoS) due to low propagation speed [16]. Based on accuracy results obtained across studies in the literature, BLE-based systems are viewed as proximity technologies compared to UWB systems which are generally more accurate, but recent improvements involving Angle of Arrival (AoA) approaches have moved the accuracy of BLE closer to that of UWB.

Though studies investigating commercial UWB-based technologies in an industrial-like environment are still scarce, a growing trend can be observed as the industry uptake of such technology rises. The authors of [7] evaluated the Pozyx Creator system in an industrial warehouse with metallic obstacles in conditions of LoS and NLoS. Pozyx [17] is based on the same DecaWaveDWM1000 chip as ATLAS in [18]. This study also aimed to improve the Pozyx proprietary positioning algorithm “Pozyx_POS_ALG_UWB_ONLY” with a multilateration algorithm. The results are based on measurements of nine ground-truth points taking 20 samples per point. The (by Pozyx) claimed accuracy of 0.1 m could be verified under LoS conditions [17] but experiments showed an average error under mixed conditions (LoS/NLoS) of 0.862 m. The standard deviation lies between 0.014 and 1.525 m depending on the area. The multilateration algorithm showed improved results of 0.51 m mean error, a 40% improvement compared to Pozyx’s algorithm. These were sufficient conditions to evaluate and contest Pozyx-claimed accuracy. The present study employs the Pozyx technology, as well.

In [16], a study compared three UWB-based commercial technologies from Ubisense, BeSpoon and DecaWave. The experiment was carried out for static measurements, conducted in an industrial warehouse containing metal structures such as vehicles, robots, metallic shelves, etc. There are both LoS and NLoS conditions. Tags are tracked across 70 reference points with a 30 s sampling time per point. The accuracy results show the mean and median for the DecaWave is 0.49 and 0.39 m, BeSpoon is 0.71 and 0.58 m and Ubisense is 1.10 and 0.61 m (w/AoA method). The study did not include a dynamic experiment to compare the performance of the systems in mobility cases. The work in [18], however, investigates the dynamic performance of ATLAS using a mobile robot following a set trajectory. The ATLAS system is based on the DWM1000 module from Decawave. OptiTrack is used as a reference system. The experiment is performed in LoS conditions and repeated 10 times. The Euclidean error of the system shows an accuracy of 0.3 m in 2D and 0.456 m in 3D in the 99% quartile. This study presents dynamic measurements of the same UWB module used in [16] in geometrical LoS-only conditions for optimal testing conditions of the ATLAS system which are not similar to industrial environments conditions.

Using another UWB-based commercial technology rather than Pozyx, the authors of [19] evaluate the TimeDomain PulsON440 in a heavy industrial galvanic environment for plating fashion accessories with precious metals. The study mentions that such a system claims an accuracy up to 2 cm. The experiment uses four beacons and one tag, describing two static tests and one dynamic in conditions of both LoS and NLoS. The tag is placed on top of a metallic frame consecutively submerged in six tanks. The first static test in LOS conditions presents the best result as 0.02 m and the worst as 0.38 m mean error. The second test in NLoS conditions (operators present) shows deteriorated results with a mean error of 0.22 m. The dynamic case illustrates the trajectory of the frame across the six tanks, as moved by the operator, implying a case of both LOS and NLoS conditions; however, statistics are not described for this case. While the authors of the study claim the commercial system to be accurate enough, descriptive statistics are important to characterize a technology in different environments.

When it comes to proximity technologies, BLE is gaining momentum in the literature. Popular commercial systems based on BLE are iBeacons from Apple Inc. and EddyStone from Google Inc., providing proximity detection in the range of 1–3 m. Moreover, positioning is mostly perfomed in 1D using the received signal strength indicator (RSSI) as a parameter for distance [11]. The authors of [20] consider BLE a good candidate for accurate IPS claiming that a trade-off between BLE’s characteristics such as sampling time and energy consumption can provide the accuracy needed. Although there are many studies on BLE as an IPS [11,21], investigations stressing this technology in industrial or harsh environments are scarce. The authors of [22] perform a static and dynamic evaluation of BLE versus Wi-Fi using 11 beacons spread across the hallways of a building floor. In the static case, BLE outperforms Wi-Fi with an average Root Mean Square (RMS) error of 3.8 m versus 5.3 m. For the dynamic case, the signal is filtered with a particle filter (PF) and the Cumulative Distribution Function (CDF) indicates an RMS error of 4 m for BLE. The work in [23] finds that different filters yield different results with BLE. A PF yields an accuracy of 1.441 m, a cascaded Kalman filter (KF)-PF yields 1.03 m and cascaded PF-Extended KF yields 0.95 m [11]. The work in [24] also compares BLE results obtained using three positioning methods for a dynamic case: dead-reckoning, trilateration and KF. The best result was 0.73 m average accuracy using the filter method. The authors of [25] track multiple workers in a confined construction site with a BLE tag fusing RSSI with KF and an accelerometer. In the static case, the average error ranges from 0.2 to 0.22 m, and in dynamic cases, the average error ranges from 0.1 to 0.47 m. The work in [26] evaluates BLE in the harsh conditions of a public transport station with highly dynamic features and NLoS. The accuracy values are similar to the ones found in the literature for environments of an office or hallways, i.e., between 0.48 and 3.6 m. The authors of [27] study accuracy differences between Wi-Fi, BLE beacons and UWB tags in a 36 m² classroom. The results show an average error of 1.39 m for Wi-Fi, 0.86 m for BLE and 0.24 m for UWB. Similar results were obtained in [28], comparing UWB and BLE and a fusion between the two. The medians of localization errors are 0.23 m for UWB and 1 m for BLE in a laboratory and elderly people’s home environments.

However, a growing number of studies are trying to achieve sub-centimetre accuracy results using methods such as fusion or Angle of Arrival/Departure (AoA/AoD). The works in [29] argue that with the arrival of new methods based on AoA/AoD, BLE can achieve more accurate measurements than those based on RSSI. The study carries static measurements of BLE 4.2 (old) and 5.0 (new) in an industrial laboratory: a classroom and a workshop. The workshop contains heavy machinery and other factory-like conditions. The best results obtained in the classroom for BLE5.0 range between ±2 m and a standard deviation of 1 m. The best results in the workshop for BLE5.0 are between 0.5 and 1.5 m of distance error. BLE4.2 showed an error between 0.5 and 1 m. In their study, the authors of [30] show that in LoS conditions in a sports hall, using AoA-only to determine distances to the device, the mean error is 0.04 m on the X-axis and 0.01 m on the Y-axis. It is noted again, as in previous studies, that as the distance between beacon and device increases, the error also increases. This error has also been shown to increase in the presence of disturbances due to multipath in NLoS conditions. AoA mitigation methods for fusing KF or IMU can be used to reduce errors. The work in [31] further investigates the feasibility of using the AoA and AoD information provided by BLE5.1 to determine the position of a device. The empirical study achieved an error of below 0.85 m for more than 95% of the positions. It shows that utilising AoA sub-meter accuracy can be achieved; however, there is still progress to be made to achieve centimetre-level accuracy.

Looking back at some of the known US-based technologies from the literature, it is obvious that systems such as BAT [32], Cricket [15] and Dolphin [10] have been used several times, e.g., in [9,10,11,12,13,33]. The Active BAT [32,34] from Olivetti and Oracle Research Laboratory (ORL) was supposed to meet the requirements of fine-granularity in office rooms which systems back then could not meet. In [11,14], the BAT system is mentioned as reaching 0.3 m in 3D. MIT’s Cricket system [15] is a location-support service by combining RF and US. The system is developed to be cheap, decentralized and provide no more than room-level precision. The authors of [11,13,33] mention the accuracy achieved by Cricket is 0.1 m in 3D and within 0.3 m 99% of the time. DOLPHIN [10]—Distributed Object Locating System for Physical-space InterNetworking—proposes to reduce configuration costs by deploying wireless sensor nodes that send and receive RF and ultrasonic signals (US). DOLPHIN has shown an accuracy of around 0.15 m using three beacons.

Novel positioning systems using US are described in [5,35,36,37,38] to name a few. In [11], the US technology is described as being able to provide fine-grained centimetre level accuracy. The work in [12] mentions that US provides an accuracy of 0.01 m with a medium-level cost and low complexity. The main disadvantages are represented by the accuracy limitation due to air temperature, humidity and pressure changes. In [36] is presented a US IPS based on a receiving node carried by the user and at least four or five transmitting nodes. It uses the Time Difference Of Arrival (TDoA) method and has an effective range of 10 m. The acclaimed accuracy of the system is 0.02 m in standard deviation. The authors of [37] developed and tested a robust indoor positioning system based on a US signal and a wireless sensor network. The measurements are evaluated against a commercial vision-based localization system with 1 mm precision. Using four anchors or beacons at a fixed position, the mobile robot moves in a circular trajectory at a very low speed in conditions of LoS. There are 4 tests carried out of different circle radii and the highest error of tracking the robot is 10.24 mm. The authors of [35] developed a low-cost FPGA-based 3D location system based on US. The location accuracy is 0.033 m in a static case. Furthermore, the system was tested on a robot used for point tracking. If the error was greater than 5 cm, the robot would reiterate the test until it reached the target point. Results showed that the robot reached the target point on the first try in 80% of cases, and in the remaining 20% of cases, it reached the target point in two attempts. Another study [5] proposes a low-cost IPS based on active US beacons for a mobile robot. US beacons are placed on the ceiling and two receivers are placed on the robot. The system measures the distance between the beacons and receivers directly. Using a Time of Flight (ToF) method for 2D positioning of the robot at a frequency of 10 Hz, the accuracy obtained is 40 mm with a standard deviation of 10 mm. Both studies [5,35] propose a new system setup for indoor localization, however, they are not thoroughly tested in NLoS or industrial conditions. A system based on the TDoA method and combining Orthogonal Frequency Division Multiplexing (OFDM) waveform with Zadoff–Chu sequences is also proposed in [39]. Performance analysis shows ranging accuracies in the order of 2 cm, with the excellent cross-correlation properties of the sequences ensuring large scalability.

In [38] is presented a comparative study between a US- and UWB-based indoor positioning systems. The systems are compared in a large indoor area (24 × 14 m or 336 m²). The US system uses a ToF method and the UWB system uses Round-Trip times (RTT) in conditions of LoS. The UWB system is a commercial product based on Decawave TREK1000 using three and six beacon nodes. The US system is provided by GEINTRA-US/RF based on their LOCATE-US prototype using three beacons. The comparison between the two systems is performed in different configurations: fixed positions, a mobile robot following a line, a person walking on a known path. In the static case, both systems presented a distance error of less than 0.2 m in 80% of cases in LoS conditions at a height of 1.5 m. The dynamic case involved a line-following robot with both systems tags on top tracking a 7.2 m long line at a height of 0.1 m. The CDF of the dynamic case showed an error of maximum 0.2 m in 80% of cases for the US system, a maximum of 0.12 m for the three-beacon UWB system and a lower than 0.08m error for the UWB system with six beacons. The test performed by a person walking a predefined path resulted in an error lower than 0.65 m in 80% of cases for the US system, whereas the results were 0.35 m for the UWB system with six beacons and an error of more than 0.5 m for the three-beacon system. When both systems have the same number of nodes, the errors are similar in movement scenarios.

A major challenge for both RF and US systems is a complex and dynamic environment due to noise, multi-path propagation and NLoS situations [13]. As noted before, a disadvantage of these systems is that their accuracy is dependent on the placement of the beacons on the ceiling or walls. Most methodologies in the literature for evaluating IPS consider LoS in environments that are less prone to interference, leading to results that cannot be generalized over multiple environments.

The value added to this field of research by our experimental 2D study is represented by:

• Comparison under the harsh conditions of an industrial environment between systems based on US and UWB;
• Large study on 100 ground-truth points and an uncontrolled environment;
• Measurements and analysis for both LoS and NLoS conditions;
• The involved methodology, including both static and dynamic cases inspired by industry: static localization for pallets and production modules, dynamic tracking of autonomous robots, forklifts, workers;
• Static measurements and analysis for different heights (0.3 m, 1 m, 2 m).

The innovation of our large measurement study stands with the extensive methodology and analysis chosen to describe the comparison. Such in-depth analysis is seldom seen in the literature for comparison studies. The analysis separates the results for static aggregate measurements across the 100 points—combining the effects of both LoS and NLoS, static single-point measurements in LoS-only conditions and dynamic measurements using filtering to reduce the static influences. We analyse the effects of obstacle density and height, as this can affect the accuracy of the tag measurements. In the static case, this is achieved by placing the tags at three heights on a tripod, as shown in Figure 1a. In the dynamic case, a mobile robot Figure 1b carries the tags and follows a trajectory through the laboratory in spaces without obstacles such as corridors, but also in-between machinery and obstacles. An involved analysis of the measurements is presented between the two cases using parameters proposed by previous studies [8,9,11,12]: accuracy and precision, cost, scalability and limitations of the technologies under assessment.