Ship detection and tracking have attracted a lot of attention in remote sensing because of the great potential in military application and port activities analysis. Compared with the vehicle targets, the size of the ship targets varies in a wide range, and the background of the track is commonly water, which may limit the performance of tracking methods. The feature of the water background is very similar to adjacent frames, which leads to ineffective motion information from the background analysis. Tracking algorithms such as optical flow-based tracker and offline tracking methods are thus not proper for ship tracking. Therefore, several novel models have been proposed to track ships from satellite videos.
1. Introduction
Object tracking is a hot topic in computer vision and remote sensing, and it typically employs a bounding box that locks onto the region of interest (ROI) when only an initial state of the target (in a video frame) is available [1][2][1,2]. Thanks to the development of satellite imaging technology, various satellites with advanced onboard cameras have been launched to obtain very high resolution (VHR) satellite videos for military and civilian applications. Compared to traditional target tracking methods, satellite video target tracking is more efficient in motion analysis and object surveillance, and has shown great potential applications in spying on enemies [3], monitoring and protecting sea ice [4], fighting wildfires [5], and monitoring city trafficking [6], which traditional target tracking cannot even approach.
Recent research has shown an increasing interest in traditional video-based target tracking, with numerous algorithms proposed for accurate tracking in computer vision. Methods that utilize generative models [7][8][9][10][7,8,9,10] or discriminant models [11][12][13][14][15][16][17][11,12,13,14,15,16,17] can be divided into two categories. The generative model-based target tracking can be thought of as a search problem, in which the object area in the current frame is modeled and the most similar region is chosen as the predicted location in the next frame. In contrast, discriminant models regard object tracking as a binary classification problem and have attracted much attention due to their efficiency and robustness [18].
The ship tracking approaches are categorized into two classes: image-based tracking methods and multi-modality-based tracking approaches. The summary of reviewed ship tracking publications is given in Table 18. In addition, Figure 14 shows a comparison of algorithm structure between two categories.
Figure 14. Comparison diagram of algorithm structure for ship tracking. (
a) the framework of Ref.
[19][100] (An Example of image-based tracking method); (
b) the procedure of track-level fusion reproduced from Ref.
[20][101] (An example of a multi-modality-based tracking method).
Table 18. Summary of the ship tracking methods.
Summary of the ship
| Target |
Method |
Ref. |
Year |
Description |
| Ship |
Image-based |
[19] | [100] |
2019 |
Automatic detection and tracking for moving ships |
| [21] | [102] |
2021 |
Framework consists of ANGS, MDDCM, JPDA |
| [22] | [103] |
2022 |
Mutual convolution SN with hierarchical double regression |
| Multi-modality |
[23] | [104] |
2010 |
Ship detection and tracking using AIS and SAR data |
| [20] | [101] |
2018 |
Track-level fusion for noncooperative ship tracking |
| [24] | [105] |
2018 |
Integrate sequential imagery with AIS data |
| [25] | [106] |
2021 |
Integrate satellite sequential imagery with ship location information |
2. Image-Based Tracking Methods
Ref.
[19][100] developed an automatic detection and tracking model for moving ships in different sizes from satellite videos, as illustrated in
Figure 14a. The dynamic multiscale saliency map was generated using motion compensation and multiscale differential saliency maps. Remote sensing images from the GO3S satellite were used to study the performance of the proposed method, indicating the effectiveness on ship tracking, especially on small ships. Furthermore, Ref.
[21][102] proposed a new framework, including
adaptive nonlinear gray stretch(ANGS
),
multiscale dual-neighbor difference contrast measure (MDDCM
), and
joint probability data association (JPDA
) methods, to detect moving ships from GF-4 satellite images
[26][137]. In Ref.
[21][102], the ANGS enhanced the image and highlighted small and dim ship targets. The MDDCM detected the position of the candidate ship target, and the JPDA was applied for multi-frame data association and tracking.
ItThe waauthors analyzed that general influencing factors on ship detection in optical remote sensing images include bright clouds and islands. In addition, high-resolution images are encouraged for better detection scores. By designing the mutual convolution Siamese network, Ref.
[22][103] calculated the similarity between the object template and the search area to enhance the significance of the ship in the feature map.
ItThe waauthors also proposed
that a a hierarchical double regression module to reduce the influence of the non-rigid motion of the water surface in the tracking phase.
3. Multi-Modality Based Tracking Methods
The automatic identification system (AIS) is an automatic tracking system that utilizes transceivers on ships and is applied by vessel traffic services. AIS information supplements marine radar, which continues to be the primary method of collision avoidance for water transport. AIS has been proven to be instrumental in accident investigation and search-and-rescue operations.
Earlier in 2010, Ref.
[23][104] studied a fused ship detection and tracking system using the AIS data and satellite-borne SAR data. A 3D extension of a standard ordered-statistics constant false alarm rate (OSCFAR) algorithm was implemented on the radar data to realize target detection. For ship tracking, an alpha-beta filter combined with a nearest neighborhood assignment strategy was proposed and performed in polar coordinates to reduce false alarm errors. A time series of 512 samples and two onboard SAR sensors were used to verify their method, showing competitive results with previous works.
Recently, there has been renewed interest in fusing optical images with AIS data. Ref.
[20][101] provided a track-level fusion architecture for GF-4 and AIS data to ship tracking tasks, as shown in
Figure 14b. The constant false alarm rate (CFAR) detector first detected ships in GF-4 images, and then the multiple hypotheses tracking (MHT) Tracker with projected AIS data was aimed to achieve ship tracking. Then,
the a
nuthors design a new track-to-track association algorithm
was designed based on iterative closest point (ICP) and global nearest neighbor (GNN) with multiple features to improve the validity of association. The core data fusion architecture was the track-to-track association based on a combined algorithm with multiple features to correct positioning errors. As reported
by the authors, their effective data fusion method showed that the AIS aided satellite image offered a great perspective for tracking non-cooperative targets. Similar to Ref.
[20][101], Ref.
[24][105] investigated the AIS aided ship-tracking method with GF-4 satellite sequential imagery. The algorithm consisted of three steps: ship detection, position correction, and ship tracking, which were realized by the peak signal-to-noise ratio (PSNR)-based local visual saliency map, the rational polynomial coefficient (RPC) model with AIS data, and amplitude assisted MHT framework, respectively. The proposed method achieved the accuracy evaluation, precision, recall, and F1-score indices with 98.5%, 87.4%, and 92.6% on GF-4 satellite sequences, indicating the accurate estimation of moving ships. In 2021, Ref.
[25][106] combined GOES-17 satellite imagery with ship location information to track the trajectories of ship-emitted aerosols based on its physical processes and optical flow model.
Encyclopedia
