Sequential Tracking Models, Physics-Based Models and Hybrid Models: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Jessie Wu and Version 1 by Md Asif Bin Syed.
Spatio-temporal, geo-referenced datasets are rapidly expanding and will continue to do so in the near future due to technological advancements as well as social and commercial factors. The introduction of the automatic identification system (AIS), which allows neighboring ships to communicate frequently with their location and navigation status via a radio signal, has enabled researchers to get their hands on datasets rich in spatio-temporal information. AIS data are collected from satellites and ground stations located all over the world. AIS data facilitates the mapping and characterization of maritime human and vessel activities, thus allowing for the real-time geo-tracking and identification of vessels equipped with AIS. Hence, in addition to its initial application in collision avoidance, AIS is now also a massive data source of unparalleled quality for diverse tracking tasks. The AIS dataset contains the location and motion features of the vessels. Each data point or row in the AIS data file is represented by a time-sequenced node that contains the vessel’s coordinates, speed, and traveling direction. Each node also has an associated time stamp indicating the data collection time. AIS dataset is suitable for  the track association problem solving approach for its spatio-temporal characteristics.
  • neural networks
  • deep learning
  • automatic identification system (AIS)
  • multi-object tracking

1. Sequential Tracking Models

Sequential tracking algorithms, such as global nearest neighbor (GNN) [12][1] and joint probabilistic data association (JPDA), [13][2], are commonly employed to update tracks based on the contribution of several objects. These algorithms utilize a cost assignment matrix to minimize costs and employ soft assignment, also known as track association probability, to achieve this goal. While global nearest neighbor (GNN) [12][1] and joint probabilistic data association (JPDA) [13][2] focus on a single hypothesis for tracking objects, there are other techniques available. For example, multiple hypotheses tracking (MHT) [14][3] constructs a tree of hypotheses for each item and computes the likelihood of each track to determine the most probable combination of tracks. The random finite set (RFS)-based approaches were also been utilized for tracking objects, as they are capable of handling the inherent uncertainty involved in the tracking process [18][4]. The majority of sequential tracking-based algorithms are based on the Kalman filtering (KF) approach [19][5] or its variations, which are frequently employed to track moving objects and provide information regarding their velocity and acceleration based on their position. However, the accuracy of KF is predicated on the assumption of linear motion, and it struggles to accommodate nonlinear motion patterns. Furthermore, the KF framework has limited capacity for handling the distinct characteristics of vessel movements.

2.2. Physics-Based Models

2. Physics-Based Models

The physics-based approaches rely on mathematical equations to describe the motion of ships, taking into account factors such as mass, force, and inertia. These equations utilize physical laws to calculate the future motion characteristics of the ship [20,21,22,23][6][7][8][9]. Such motion models can be useful for developing simulation systems to study ideal ship kinematic characteristics or even to train navigation systems. However, applying these models to track the trajectory patterns of multiple ships can be challenging. While these models can incorporate the spatio-temporal patterns of vessel movements [1][10] in the learning process, they are still limited in nature, only considering the last known position to track vessels.

2.3. Machine Learning and Hybrid Models

3. Machine Learning and Hybrid Models

These methods rely solely on historical data and employ machine learning techniques to learn from past information, enabling them to predict future positions when provided with a new feature vector. These prominent machine learning methods used in trajectory prediction studies include the Gaussian process, support vector machine, principal component analysis (PCA), etc. While these methods [24,25,26,27,28][11][12][13][14][15] typically perform well in predicting immediate future positions, their prediction accuracy tends to decrease as the prediction time span increases. Furthermore, the performance of these models is highly dependent on the proper tuning of hyperparameters, which can be difficult to achieve. Additionally, they are not capable of processing long sequences and unraveling the spatial and temporal dependencies present in sequential observations. Hybrid approaches, on the other hand, combine physics-based models and machine learning models or different machine learning models to enhance the quality of the trajectory tracking process [29,30,31,32,33,34][16][17][18][19][20][21]. These approaches, however, are not free from the limitations imposed by the physics-based and machine learning models.

2.4. Deep Learning-Based Models

4. Deep Learning-Based Models

Deep learning, which is a subclass of machine learning models, stands out from the rest due to its superior learning capabilities. In the context of marine vessel trajectories, neural networks have been widely used for their ability to process large datasets and discover long-term patterns hidden in vessel trajectories.Because of the robust adaptability, the earliest form of neural network, including the multi-layer perceptron (MLP) [35][22] and artificial neural network (ANN) [36,37][23][24] played a significant role in traffic and marine vessel trajectory prediction. Nevertheless, despite their wide applications, these neural networks exhibit low interpretability. Additionally, they present substantial challenges in terms of spatial and temporal information processing capability [38][25] since these networks are not equipped to handle such characteristics.
The exploration of incorporating sequential temporal patterns into marine ship trajectory prediction has motivated researchers to investigate the potential application of recurrent neural networks (RNNs) [19][5]. However, RNNs encounter challenges in capturing long-term dependencies within a sequence due to the issue of vanishing gradients during backpropagation [39][26]. As a result, the limited long-term memory of these networks can hinder their performance when the data contain significant long-term dependencies [5,40,41][27][28][29]. Two prominent variations of recurrent neural networks (RNNs), specifically long short-term memory (LSTM) [42,43][30][31] and gated recurrent unit (GRU) [44][32], have garnered substantial attention for their remarkable ability to uncover underlying patterns within extended input sequences, proving particularly advantageous for trajectory prediction.
Further advancements in research have led to the utilization of more efficient variants of LSTM and GRU, such as bidirectional LSTM (Bi-LSTM) [4[33][34],45], bidirectional GRU (Bi-GRU) [46][35], context-aware LSTM (C-LSTM) model [47][36], and multi-step prediction LSTM (MP-LSTM) [48][37]. Distinct from traditional LSTM, Bi-LSTM has the ability to process data from both past and future contexts. This bidirectional information processing, encompassing both forward and backward information, empowers Bi-LSTM to capture a comprehensive understanding of the sequence. Consequently, numerous innovative models based on Bi-LSTM have been proposed for ship trajectory prediction. However, these models often exhibit significant computational complexity and limited generalization capabilities. The design parameters of these neural network-based frameworks are adjusted in real-time as the vessel progresses, enabling them to identify all potential trajectories a vessel may follow and reconstruct (predict) its trajectories for future time points [37][24]. Additionally, LSTM networks have demonstrated remarkable multitasking performance [49][38].
Another neural network, the convolutional neural network (CNN), originally devised to address computer vision problems, has also been explored for the trajectory prediction and classification of the tracks [49][38] as it can help capture the spatial patterns exist in the trajectory data. Instead of using the original features, several methods advocate the use of latent features derived from the neural network architecture. These methods leverage latent space representation using variational recurrent autoencoder (VRAE) [50][39] or LSTM [51][40]. These latent features can capture the spatial patterns present in the data. Temporal ordering and attention maps are also proven to be effective for object tracking [52][41].
Instead of using the original features, several methods advocate the use of latent features derived from the neural network architecture. These methods leverage latent space representation using variational recurrent autoencoder (VRAE) [50][39] or LSTM [51][40]. These latent features can capture the spatial patterns present in the data. Temporal ordering and attention maps are also proven to be effective for object tracking [52][41].
In addition to conventional deep learning approaches, the research field has expanded to include the application of hybrid deep learning architectures directly to raw datasets. This advancement goes beyond transforming data into a latent space and aims to reveal both temporal and spatial relationships among features. It has proven to be particularly effective in extracting spatio-temporal relationships within the AIS dataset. Hybrid deep learning-based models for ship trajectory prediction, such as the integration of bidirectional LSTM and RNN (BLSTM-RNN) [53][42] and CNN-LSTM-SE [54][43], have emerged as notable techniques due to their rapid learning and adaptability capabilities. These approaches excel in producing highly accurate results when dealing with complex and dynamic trajectory data.
However, it is important to note that these methods primarily focus on predicting the next points by considering the sequence of vessel nodes. This differs from track association, which aims to link vessels to their respective tracks. Furthermore, following the prediction route would require a separate prediction model for each vessel, which can complicate the tracking process when dealing with more than ten vessels. The proposed 1D CNN-LSTM model can overcome all these issues and classify multiple vessels by capturing the spatial and temporal patterns hidden in the data.

References

  1. Blackman, S.S. Multiple hypothesis tracking for multiple target tracking. IEEE Aerosp. Electron. Syst. Mag. 2004, 19, 5–18.
  2. Bar-Shalom, Y.; Li, X.R. Multitarget-Multisensor Tracking: Principles and Techniques; YBs: Storrs, CT, USA, 1995; Volume 19.
  3. Reid, D. An algorithm for tracking multiple targets. IEEE Trans. Autom. Control 1979, 24, 843–854.
  4. Pang, S.; Radha, H. Multi-Object Tracking Using Poisson Multi-Bernoulli Mixture Filtering For Autonomous Vehicles. In Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada, 6–11 June 2021; pp. 7963–7967.
  5. Capobianco, S.; Millefiori, L.M.; Forti, N.; Braca, P.; Willett, P. Deep learning methods for vessel trajectory prediction based on recurrent neural networks. IEEE Trans. Aerosp. Electron. Syst. 2021, 57, 4329–4346.
  6. Best, R.A.; Norton, J. A new model and efficient tracker for a target with curvilinear motion. IEEE Trans. Aerosp. Electron. Syst. 1997, 33, 1030–1037.
  7. Caveney, D. Numerical integration for future vehicle path prediction. In Proceedings of the 2007 American Control Conference, New York, NY, USA, 9–13 July 2007; pp. 3906–3912.
  8. Semerdjiev, E.; Mihaylova, L. Variable-and fixed-structure augmented interacting multiple model algorithms for manoeuvring ship tracking based on new ship models. Int. J. Appl. Math. Comput. Sci. 2000, 10, 591–604.
  9. Khan, A.; Bil, C.; Marion, K.E. Ship motion prediction for launch and recovery of air vehicles. In Proceedings of the OCEANS 2005 MTS/IEEE, Washington, DC, USA, 17–23 September 2005; pp. 2795–2801.
  10. Ahmed, I.; Jun, M.; Ding, Y. A Spatio-Temporal Track Association Algorithm Based on Marine Vessel Automatic Identification System Data. IEEE Trans. Intell. Transp. Syst. 2022, 23, 20783–20797.
  11. Bartelmaos, S.; Abed-Meraim, K.; Attallah, S. Fast algorithms for minor component analysis. In Proceedings of the IEEE/SP 13th Workshop on Statistical Signal Processing, Bordeaux, France, 17–20 July 2005; pp. 239–244.
  12. Peng, D.; Yi, Z. A new algorithm for sequential minor component analysis. Int. J. Comput. Intell. Res. 2006, 2, 207–215.
  13. Simsir, U.; Ertugrul, S. Prediction of manually controlled vessels’ position and course navigating in narrow waterways using Artificial Neural Networks. Appl. Soft Comput. 2009, 9, 1217–1224.
  14. Joseph, J.; Doshi-Velez, F.; Huang, A.S.; Roy, N. A Bayesian nonparametric approach to modeling motion patterns. Auton. Robot. 2011, 31, 383.
  15. Pallotta, G.; Horn, S.; Braca, P.; Bryan, K. Context-enhanced vessel prediction based on Ornstein-Uhlenbeck processes using historical AIS traffic patterns: Real-world experimental results. In Proceedings of the 17th International Conference on Information Fusion (FUSION), Salamanca, Spain, 7–10 July 2014; pp. 1–7.
  16. Guo, X.R.; Wang, F.H.; Du, D.F.; Guo, X.L. An improved neural network based fuzzy self-adaptive Kalman filter and its application in cone picking robot. In Proceedings of the 2009 International Conference on Machine Learning and Cybernetics, Baoding, China, 12–15 July 2009; Volume 1, pp. 573–577.
  17. Perera, L.P.; Soares, C.G. Ocean vessel trajectory estimation and prediction based on extended Kalman filter. In Proceedings of the Second International Conference on Adaptive and Self-Adaptive Systems and Applications, Lisbon, Portugal, 21–26 November 2010; pp. 14–20.
  18. Stateczny, A.; Kazimierski, W. Multisensor Tracking of Marine Targets: Decentralized Fusion of Kalman and Neural Filters. Int. J. Electron. Telecommun. 2011, 57, 65–70.
  19. Perera, L.P.; Oliveira, P.; Soares, C.G. Maritime traffic monitoring based on vessel detection, tracking, state estimation, and trajectory prediction. IEEE Trans. Intell. Transp. Syst. 2012, 13, 1188–1200.
  20. Dalsnes, B.R.; Hexeberg, S.; Flåten, A.L.; Eriksen, B.O.H.; Brekke, E.F. The neighbor course distribution method with Gaussian mixture models for AIS-based vessel trajectory prediction. In Proceedings of the 2018 21st International Conference on Information Fusion (FUSION), Cambridge, UK, 10–13 July 2018; pp. 580–587.
  21. Tu, E.; Zhang, G.; Mao, S.; Rachmawati, L.; Huang, G.B. Modeling Historical AIS Data for Vessel Path Prediction: A Comprehensive Treatment. arXiv 2020, arXiv:2001.01592.
  22. Valsamis, A.; Tserpes, K.; Zissis, D.; Anagnostopoulos, D.; Varvarigou, T. Employing traditional machine learning algorithms for big data streams analysis: The case of object trajectory prediction. J. Syst. Softw. 2017, 127, 249–257.
  23. Chen, R.; Chen, M.; Li, W.; Guo, N. Predicting future locations of moving objects by recurrent mixture density network. ISPRS Int. J. Geo-Inf. 2020, 9, 116.
  24. Volkova, T.A.; Balykina, Y.E.; Bespalov, A. Predicting ship trajectory based on neural networks using AIS data. J. Mar. Sci. Eng. 2021, 9, 254.
  25. Li, H.; Jiao, H.; Yang, Z. AIS data-driven ship trajectory prediction modelling and analysis based on machine learning and deep learning methods. Transp. Res. Part E Logist. Transp. Rev. 2023, 175, 103152.
  26. Bengio, Y.; Simard, P.; Frasconi, P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 1994, 5, 157–166.
  27. Tang, H.; Yin, Y.; Shen, H. A model for vessel trajectory prediction based on long short-term memory neural network. J. Mar. Eng. Technol. 2022, 21, 136–145.
  28. Borkowski, P. The ship movement trajectory prediction algorithm using navigational data fusion. Sensors 2017, 17, 1432.
  29. Zheng, H.; Negenborn, R.R.; Lodewijks, G. Trajectory tracking of autonomous vessels using model predictive control. IFAC Proc. Vol. 2014, 47, 8812–8818.
  30. Hammedi, W.; Brik, B.; Senouci, S.M. Toward optimal MEC-based collision avoidance system for cooperative inland vessels: A federated deep learning approach. IEEE Trans. Intell. Transp. Syst. 2022, 24, 2525–2537.
  31. Karataş, G.B.; Karagoz, P.; Ayran, O. Trajectory pattern extraction and anomaly detection for maritime vessels. Internet Things 2021, 16, 100436.
  32. Suo, Y.; Chen, W.; Claramunt, C.; Yang, S. A ship trajectory prediction framework based on a recurrent neural network. Sensors 2020, 20, 5133.
  33. Park, J.; Jeong, J.; Park, Y. Ship trajectory prediction based on bi-LSTM using spectral-clustered AIS data. J. Mar. Sci. Eng. 2021, 9, 1037.
  34. Yang, C.H.; Wu, C.H.; Shao, J.C.; Wang, Y.C.; Hsieh, C.M. AIS-based intelligent vessel trajectory prediction using bi-LSTM. IEEE Access 2022, 10, 24302–24315.
  35. Wang, C.; Ren, H.; Li, H. Vessel trajectory prediction based on AIS data and bidirectional GRU. In Proceedings of the 2020 International Conference on Computer Vision, Image and Deep Learning (CVIDL), Chongqing, China, 10–12 July 2020; pp. 260–264.
  36. Mehri, S.; Alesheikh, A.A.; Basiri, A. A contextual hybrid model for vessel movement prediction. IEEE Access 2021, 9, 45600–45613.
  37. Gao, D.w.; Zhu, Y.s.; Zhang, J.f.; He, Y.k.; Yan, K.; Yan, B.r. A novel MP-LSTM method for ship trajectory prediction based on AIS data. Ocean Eng. 2021, 228, 108956.
  38. Wang, W.; Bin, J.; Zaji, A.; Halldearn, R.; Guillaume, F.; Li, E.; Liu, Z. A multi-task learning-based framework for global maritime trajectory and destination prediction with AIS data. Marit. Transp. Res. 2022, 3, 100072.
  39. Murray, B.; Perera, L.P. An AIS-based deep learning framework for regional ship behavior prediction. Reliab. Eng. Syst. Saf. 2021, 215, 107819.
  40. Yang, C.H.; Lin, G.C.; Wu, C.H.; Liu, Y.H.; Wang, Y.C.; Chen, K.C. Deep Learning for Vessel Trajectory Prediction Using Clustered AIS Data. Mathematics 2022, 10, 2936.
  41. Xu, Y.; Zhou, X.; Chen, S.; Li, F. Deep learning for multiple object tracking: A survey. IET Comput. Vis. 2019, 13, 355–368.
  42. Zhong, C.; Jiang, Z.; Chu, X.; Liu, L. Inland ship trajectory restoration by recurrent neural network. J. Navig. 2019, 72, 1359–1377.
  43. Wang, X.; Xiao, Y. A Deep Learning Model for Ship Trajectory Prediction Using Automatic Identification System (AIS) Data. Information 2023, 14, 212.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback