Robot Localization Methods: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Sirius Huang and Version 1 by Jose Luis Sanchez-Lopez.

Robots are starting to be used on construction sites to perform various tasks. Robots utilizing only geometric information for localization within a construction site may not be able to localize correctly due to the fact the ongoing site does not resemble yet the architectural plans. Semantic information can help improve the localization but still can suffer inaccuracies due to large deviation of the robot pose estimate from its actual estimate and similar semantic information in case of repetitive environments.

  • localization
  • SLAM
  • Building Information Modelling (BIM)

1. Introduction

Robots are starting to be used on construction sites to perform various tasks such as autonomous data capturing to continuously monitor the evolution of the construction site, while construction environments are challenging because of their dynamic nature, as they can change significantly over a span of a few days, they have some features that are common in most of them, for example, walls, rooms, or pillars. Moreover, many companies are now integrating digital tools such as Building Information Modelling (BIM) [1] in their project design and execution phases. BIM contains geometric, semantic, topological, and physical information about each and every element of the building. Such crucial information is invaluable for a robot, which can use this knowledge for localization and understanding the construction site around it. Nevertheless, nowadays robots are unable to exploit the power of these digital plans. When they navigate in the environment, they either build their internal maps from scratch, which makes it difficult for them to know where they are, or they require fiducial markers to localize properly.
Most of the ongoing construction sites can be well represented using walls and rooms, to provide the robot with pre-existing knowledge of the environment. Robots utilizing only geometric information for localization within a construction site may not be able to localize correctly due to the fact the ongoing site does not resemble yet the architectural plans. Semantic information can help improve the localization but still can suffer inaccuracies due to large deviation of the robot pose estimate from its actual estimate and similar semantic information in case of repetitive environments. Thus, research is still required to combine the complete geometric, semantic, and topological information provided by the BIM models, to robustly localize a robot within an ongoing construction environment.

2. Global Localization

Global localization involves determining the position of a robot on a map that is already known. It is typically done in two steps. The first step is initial localization, which involves matching sensor readings to the map. This is sometimes called the kidnapped robot prolem [3][2], re-localization [4][3], or place recognition [5][4]. Multiple sensor modalities, including LiDAR [6][5], monocular [7][6], and stereo [8][7] cameras, can be employed for such tasks. One of the initial studies in this area was conducted by Fox et al. [9][8], who utilized a Markov model to update the probability distribution over potential locations in the environment based on sensor readings. Another method proposed by [10][9], called Monte Carlo Localization (MCL), models the robot state as a group of particles randomly sampled from the probability density function. MCL has been found to be highly effective in various situations [11,12,13,14][10][11][12][13]. Koide et al. [15][14] proposed global localization using 3d LiDAR measurements based on an Unscented Kalman Filter (UKF). Alternative methods such as [16,17,18][15][16][17] utilize the extraction and matching of local point cloud descriptors. Key-point-based approaches face limitations due to the restricted uniqueness of local structures in LiDAR data. Nevertheless, they have gained traction as a substitute for those based on scan matching, attracting significant interest from the research community in recent years [19,20][18][19]. Learning-based localization has also shown promise in recent research. One example is SegMap, as presented in [21][20], which employs a distinct map representation for localization by extracting segments from 3D point clouds.

3. Architectural Plans Based Localization

In robotics use of 2D CAD-based models is very common to generate a 2D map of the environment and to localize a robot using the on-board range sensors [22,23][21][22]. Ref. [24][23] utilizes a monocular system to localize the robot in the 2D floor plans extracting the room layout edges. Refs. [25,26][24][25] incorporate semantic textual cues into the 2D floor plans to further improve the localization accuracy. BIM on the other hand provides additional semantic and topological information as compared to 2D CAD plans which can be exploited to improve the planning and localization of mobile robots. Authors in [27,28,29][26][27][28] utilize BIM information for extracting 3D maps utilized by path-planner but do not include this information for robot localization. Authors in [30][29] only present a small case study localizing the robot through image matching between real images and images from BIM plans, but only restricted to a certain perspective. Ref. [31][30] present an approach to extracting information from BIM, splitting it into different floor levels, and then converting it to a relevant format used by a state-of-the-art pose-graph SLAM algorithm. Since the pose-graph algorithm cannot take as input an entire 3D mesh of the area, the authors have to create an additional module to generate trajectories within the mesh storing pose and sensor data at different intervals to later input it the SLAM for localization. Whereas in [32][31], authors present a case study converting BIM into 3D meshes at different floor levels and using a simple Iterative Closest Point (ICP) algorithm to localize the robot, thus depending highly on the metric ICP algorithm for convergence, which can show inaccuracies if an on-going construction site does not yet resemble the BIM plans. Compared to previous approaches, ref. [33][32] extract semantic information in addition to geometric information from the BIM plans to localize the robot using an on-board 2D LiDAR and pose-graph optimization, but the authors do not consider additional topological information connecting different walls to a room, etc.

4. Scene Graph Based Localization

Scene graphs are emerging research to represent the scene in several geometric, semantic, and topological dimensions [34,35,36,37,38][33][34][35][36][37]. S-Graphs [2][38] is a real-time optimizable hierarchical scene graph that represents the environment in geometric, semantic, and topological dimensions. S-Graphs is capable of not only representing the environment as a scene graph but also simultaneously optimizing all the elements within it including the robot pose. All of these methods need the robot to navigate the environment in order to create the scene graphs, which means they need to know the starting point of the robot beforehand. They lack the capability of performing global localization. Some methods have also emerged exploiting scene graphs for localization such as [39,40,41][39][40][41] performing graph-based localization utilizing spatial information between the neighboring landmarks. Similarly, authors in [42] create a global topological semantic map of the environment and later use similarity between semantic scene graphs to localize the robot. Ref. [43] combine traditional 3D object alignment with graph-based matching to perform global localization in small-sized areas such as rooms.

References

  1. Azhar, S.; Hein, M.; Sketo, B. Building information modeling (BIM): Benefits, risks and challenges. In Proceedings of the 44th ASC Annual Conference, Halifax, NS, Canada, 22–26 September 2008; pp. 2–5.
  2. Desrochers, B.; Lacroix, S.; Jaulin, L. Set-membership approach to the kidnapped robot problem. In Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 28 September–2 October 2015; pp. 3715–3720.
  3. Kendall, A.; Cipolla, R. Modelling uncertainty in deep learning for camera relocalization. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 16–21 May 2016; pp. 4762–4769.
  4. Lowry, S.; Sünderhauf, N.; Newman, P.; Leonard, J.J.; Cox, D.; Corke, P.; Milford, M.J. Visual place recognition: A survey. IEEE Trans. Robot. 2015, 32, 1–19.
  5. Miller, I.D.; Cowley, A.; Konkimalla, R.; Shivakumar, S.S.; Nguyen, T.; Smith, T.; Taylor, C.J.; Kumar, V. Any way you look at it: Semantic crossview localization and mapping with lidar. IEEE Robot. Autom. Lett. 2021, 6, 2397–2404.
  6. Leung, K.Y.K.; Clark, C.M.; Huissoon, J.P. Localization in urban environments by matching ground level video images with an aerial image. In Proceedings of the 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA, 19–23 May 2008; pp. 551–556.
  7. Zuo, X.; Ye, W.; Yang, Y.; Zheng, R.; Vidal-Calleja, T.; Huang, G.; Liu, Y. Multimodal localization: Stereo over LiDAR map. J. Field Robot. 2020, 37, 1003–1026.
  8. Fox, D.; Burgard, W.; Thrun, S. Markov Localization for Reliable Robot Navigation and People Detection. J. Artif. Intell. Res. 1999, 11, 391–427.
  9. Dellaert, F.; Fox, D.; Thrun, S.; Burgard, W. Monte Carlo Localization for Mobile Robots. 1999. Available online: https://www.ri.cmu.edu/pub_files/pub1/dellaert_frank_1999_2/dellaert_frank_1999_2.pdf (accessed on 28 February 2023).
  10. Fox, D. KLD-sampling: Adaptive particle filters. In Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada, 3–8 December 2001; Volume 14.
  11. Röwekämper, J.; Sprunk, C.; Tipaldi, G.D.; Stachniss, C.; Pfaff, P.; Burgard, W. On the position accuracy of mobile robot localization based on particle filters combined with scan matching. In Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Algarve, Portugal, 7–12 October 1999.
  12. Thrun, S.; Fox, D.; Burgard, W. Monte Carlo localization with mixture proposal distribution. In Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on on Innovative Applications of Artificial Intelligence, Austin, TX, USA, 30 July–3 August 2000; pp. 859–865.
  13. Saarinen, J.; Andreasson, H.; Stoyanov, T.; Lilienthal, A.J. Normal Distributions Transform Monte-Carlo Localization (NDT-MCL). In Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Tokyo, Japan, 3–7 November 2013.
  14. Koide, K.; Miura, J.; Menegatti, E. A portable three-dimensional LIDAR-based system for long-term and wide-area people behavior measurement. Int. J. Adv. Robot. Syst. 2019, 16, 1729881419841532.
  15. Rusu, R.; Blodow, N.; Beetz, M. Fast point feature histograms (fpfh) for 3d registration. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, 12–17 May 2009.
  16. Tombari, F.; Salti, S.; Stefano, L.D. Unique signatures of histograms for local surface description. In Proceedings of the 11th European Conference on Computer Vision, Heraklion, Greece, 5–11 September 2010.
  17. Steder, B.; Ruhnke, M.; Grzonka, S.; Burgard, W. Place recognition in 3d scans using a combination of bag of words and point feature based relative pose estimation. In Proceedings of the 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Francisco, CA, USA, 25–30 September 2011.
  18. Rhling, T.; Mack, J.; Schulz, D. Fast histogram-based similarity measure for detecting loop closures in 3-d lidar data. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 28 September–3 October 2015.
  19. He, L.; Wang, X.; Zhang, H. M2dp: A novel 3d point cloud descriptor and its application in loop closure detection. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Republic of Korea, 9–14 October 2016.
  20. Dubé, R.; Cramariuc, A.; Dugas, D.; Sommer, H.; Dymczyk, M.; Nieto, J.; Siegwart, R.; Cadena, C. SegMap: Segment-based mapping and localization using data-driven descriptors. Int. J. Robot. Res. 2020, 39, 339–355.
  21. Park, Y.S.; Kim, J.; Kim, A. Radar Localization and Mapping for Indoor Disaster Environments via Multi-modal Registration to Prior LiDAR Map. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 3–8 November 2019; pp. 1307–1314.
  22. Li, Z.; Ang, M.H.; Rus, D. Online Localization with Imprecise Floor Space Maps using Stochastic Gradient Descent. In Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 25–29 October 2020; pp. 8571–8578.
  23. Boniardi, F.; Valada, A.; Mohan, R.; Caselitz, T.; Burgard, W. Robot Localization in Floor Plans Using a Room Layout Edge Extraction Network. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 3–8 November 2019.
  24. Cui, L.; Rong, C.; Huang, J.; Rosendo, A.; Kneip, L. Monte-Carlo Localization in Underground Parking Lots using Parking Slot Numbers. In Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, 27 September–1 October 2021; pp. 2267–2274.
  25. Zimmerman, N.; Wiesmann, L.; Guadagnino, T.; Läbe, T.; Behley, J.; Stachniss, C. Robust Onboard Localization in Changing Environments Exploiting Text Spotting. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan, 23–27 October 2022.
  26. Siemiatkowska, B.; Harasymowicz-Boggio, B.; Przybylski, M.; Rozanska-Walczuk, M.; Wisniowski, M.; Kowalski, M. BIM Based Indoor Navigation System of Hermes Mobile Robot. In Proceedings of the 19th CISM-Iftomm Symposium; Springer: Berlin/Heidelberg, Germany, 2013.
  27. Park, J.; Cho, Y.K.; Martinez, D. A BIM and UWB integrated mobile robot navigation system for indoor position tracking applications. J. Constr. Eng. Proj. Manag. 2016, 6, 30–39.
  28. Xu, M.; Wei, S.; Zlatanova, S.; Zhang, R. BIM-based indoor path planning considering obstacles. ISPRS Ann. Photogramm. Remote. Sens. Spat. Inf. Sci. 2017, 4, 417.
  29. Asadi, K.; Ramshankar, H.; Noghabaei, M.; Han, K. Real-time image localization and registration with BIM using perspective alignment for indoor monitoring of construction. J. Comput. Civ. Eng 2019, 33, 04019031.
  30. Moura, M.S.; Rizzo, C.; Serrano, D. BIM-based Localization and Mapping for Mobile Robots in Construction. In Proceedings of the 2021 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), Santa Maria da Feira, Portugal, 28–29 April 2021; pp. 12–18.
  31. Yin, H.; Liew, J.M.; Lee, W.L.; Ang, M.H.; Yeoh, K.W.-J. Towards BIM-based robot localization: A real-world case study. In Proceedings of the 39th International Symposium on Automation and Robotics in Construction, Bogota, Colombia, 12–15 July 2022.
  32. Hendrikx, R.W.M.; Pauwels, P.; Torta, E.; Bruyninckx, H.J.; van de Molengraft, M.J.G. Connecting Semantic Building Information Models and Robotics: An application to 2D LiDAR-based localization. In Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi’an, China, 30 May–5 June 2021; pp. 11654–11660.
  33. Kim, U.; Park, J.; Song, T.; Kim, J. 3-D Scene Graph: A Sparse and Semantic Representation of Physical Environments for Intelligent Agents. IEEE Trans. Cybern. 2019, 50, 4921–4933.
  34. Armeni, I.; He, Z.Y.; Gwak, J.; Zamir, A.R.; Fischer, M.; Malik, J.; Savarese, S. 3D Scene Graph: A structure for unified semantics, 3D space, and camera. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 5664–5673.
  35. Wu, S.C.; Wald, J.; Tateno, K.; Navab, N.; Tombari, F. SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequences. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021.
  36. Rosinol, A.; Gupta, A.; Abate, M.; Shi, J.; Carlone, L. 3D dynamic scene graphs: Actionable spatial perception with places, objects, and humans. arXiv 2020, arXiv:2002.06289.
  37. Hughes, N.; Chang, Y.; Carlone, L. Hydra: A Real-time Spatial Perception Engine for 3D Scene Graph Construction and Optimization. arXiv 2022, arXiv:2201.13360.
  38. Bavle, H.; Sanchez-Lopez, J.L.; Shaheer, M.; Civera, J.; Voos, H. Situational Graphs for Robot Navigation in Structured Indoor Environments. IEEE Robot. Autom. Lett. 2022, 7, 9107–9114.
  39. Qin, C.; Zhang, Y.; Liu, Y.; Lv, G. Semantic loop closure detection based on graph matching in multi-objects scenes. J. Vis. Commun. Image Represent. 2021, 76, 103072.
  40. Gawel, A.; Del Don, C.; Siegwart, R.; Nieto, J.; Cadena, C. X-View: Graph-Based Semantic Multi-View Localization. IEEE Robot. Autom. Lett. 2018, 3, 1687–1694.
  41. Guo, X.; Hu, J.; Chen, J.; Deng, F.; Lam, T.L. Semantic Histogram Based Graph Matching for Real-Time Multi-Robot Global Localization in Large Scale Environment. IEEE Robot. Autom. Lett. 2021, 6, 8349–8356.
  42. Wang, F.; Zhang, C.; Tang, F.; Jiang, H.; Wu, Y.; Liu, Y. Lightweight Object-level Topological Semantic Mapping and Long-term Global Localization based on Graph Matching. arXiv 2022, arXiv:2201.05977.
  43. Liu, Y.; Petillot, Y.; Lane, D.; Wang, S. Global Localization with Object-Level Semantics and Topology. In Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada, 20–24 May 2019; pp. 4909–4915.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback