Typhoon Tracking Based on Satellite Videos: Comparison
Please note this is a comparison between Version 1 by Zhaoxiang Zhang and Version 3 by Amina Yu.

Object tracking is a hot topic in computer vision and remote sensing, and it typically employs a bounding box that locks onto the region of interest (ROI) when only an initial state of the target (in a video frame) is available. The rapid development of remote sensing technologies provides a new methodology for weather observation and forecasting tasks using high-resolution visual data. A growing body of literature investigating the deep neural network-based cyclone track prediction from satellite imagery sequences has been published.

  • satellite video
  • traffic target tracking
  • typhoon tracking

1. Introduction

Recent research has shown an increasing interest in traditional video-based target tracking, with numerous algorithms proposed for accurate tracking in computer vision. Methods that utilize generative models [1][2][3][4][7,8,9,10] or discriminant models [5][6][7][8][9][10][11][11,12,13,14,15,16,17] can be divided into two categories. The generative model-based target tracking can be thought of as a search problem, in which the object area in the current frame is modeled and the most similar region is chosen as the predicted location in the next frame. In contrast, discriminant models regard object tracking as a binary classification problem and have attracted much attention due to their efficiency and robustness [12][18]. A classifier is used and trained for discriminant models, with the attributes of the object and background labeled as positive and negative samples in the current frame. In the following frame, the classifier is used to identify the foreground, and the results are updated.

Typhoon tracking methods are reviewed and divided into three classes, including the convolutional neural network (CNN)-based models, Generative Adversarial Network (GAN)AN-based models, and (recurrent neural network) recurrent neural network (RNN)RNN-based models. Figure 15 visualizes the three structures of CNN, GAN, and RNN-based typhoon tracking models. The summary of reviewed typhoon tracking publications is given in Table 18.
Figure 15. Comparison diagram of algorithm structure for (a) CNN-based, (b) GAN-based, and (c) RNN-based (specifically CLSTM) typhoon tracking.
Table 18.
Summary of typhoon tracking methods.
Target
Typhoon
Method
CNN-based
Ref.
[107]
Year
2017
Description
A multi-layer model for multichannel image sequences
[108]2020A quasi-supervised mask region CNN
GAN-based[109]2019GAN to track and predict typhoon motion
[110]2021GAN with deep multi-scale frame prediction method
[111]2022GAN to predict both the track and intensity of typhoons
RNN-based[112]2017A convolutional sequence-to-sequence autoencoder
[113]2018MNNs for typhoon tracking
[114]2018A CLSTM based model
[115]2021A CLSTM layer with FCLs
[116]2022A CLSTM with 3D CNN based on multimodal data
[117]2022An echo state network-based tracking
TyphoonCNN-based[13]2017A multi-layer model for multichannel image sequences
[14]2020A quasi-supervised mask region CNN
GAN-based[15]2019GAN to track and predict typhoon motion
[16]2021GAN with deep multi-scale frame prediction method
[17]2022GAN to predict both the track and intensity of typhoons
RNN-based[18]2017A convolutional sequence-to-sequence autoencoder
[19]2018MNNs for typhoon tracking
[20]2018A CLSTM based model
[21]2021A CLSTM layer with FCLs
[22]2022A CLSTM with 3D CNN based on multimodal data
[23]2022An echo state network-based tracking

2. CNN-Based Tracking Methods

2. CNN-Based Tracking Methods

To understand complex atmospheric dynamics based on multichannel 3D satellite image sequences, Ref. [13][107] introduced a multi-layer neural network. Multiple convolutional layers were first formed for typhoon feature extraction, followed by multiple fully connected dense layers with linear activation for linear metrics regression. In the regression step, the pixel related to the weather event was chosen as the target value. The proposed model was studied by a 2674-image satellite dataset acquired by the COMS-1 meteorological imagery [24][139], achieving a Root Mean Squared Error (RMSE) of ~0.02 to predict the center of a single typhoon that represented ~74.53 km in great circle distance. As the authors presented, a CNN could predict the coordinates of single typhoons efficiently, while the multiple typhoon case and unsupervised sequences of images needed to be further investigated. By further exploring the potential of the CNN models in cyclone detection, Ref. [14][108] designed a quasi-supervised mask region CNN. The seasonal march and spatial distribution of cyclone frequencies were derived from the proposed model. Compared with traditional methods, the presented method increased the number of identified cyclones by 8.29%, showing its good performance in identifying the horizontal structures of tropical cyclones.

3. GAN-Based Tracking Methods

Models such as those above can be categorized as discriminative models as they use conditional probability to predict the unseen data, while other methods employ generative models that make predictions by modeling joint distribution and are capable of generating new data. For example, Ref. [15][109] introduced a GAN to track and predict the typhoon centers and future cloud appearance simultaneously. A typical GAN structure was trained in an adversarial way to generate a 6-hour-advance track of a typhoon. The predicted typhoon track favorably identified the future typhoon location and the deformed cloud structures. The achieved averaged difference error between the predicted and ground truth typhoon centers was 95.6 km by calculating ten typhoon datasets. The tracking prediction could be significantly improved when employing both velocity fields and satellite images to deal with sudden changes in the track. Later, Ref. [16][110] integrated the GAN model with a deep multi-scale frame prediction algorithm, aiming to predict the atmospheric motion vectors of typhoons. The experiment results illustrated that the generated atmospheric motion vectors depicted the structure of typhoon atmospheric circulations with a certain level of accuracy. Similarly, Ref. [17][111] designed a GAN based approach to predict both the track and intensity of typhoons for short lead times within fractions of a second. The experimental results indicated that learning velocity, temperature, pressure, and humidity along with satellite images have positive effects on trajectory prediction accuracy.

4. RNN-Based Tracking Methods

Another idea dealing with tracking tasks focuses on RNN models, which have shown promising performance in processing the time series data in various areas. Ref. [18][112] developed a convolutional sequence-to-sequence autoencoder in 2017 to predict the undiscovered weather situations from satellite image series. In 2018, Ref. [19][113] presented (matrix neural network) MNNs to predict cyclone tracks for satellite imagery sequences from the South Indian Ocean area. The MNNs were trained based on matrix convolutional units and utilized to propagate the information from the input matrix to the output layer. A dataset consisting of 286 cyclones was used to verify the effectiveness of the matrix neural network (MNNs)MNNs in typhoon tracking. In the same year, Ref. [20][114] designed a convolutional Long short-term memory (LSTM) model to track and predict the tropical cyclone path. In their experiments, the proposed approach was successful in learning the spatiotemporal dynamics of the atmosphere.
In 2021, Ref. [21][115] compared various CNN and RNN recognition algorithms and proposed that the best performing network implemented a convolutional LSTM layer with fully connected layer (FCLs). Cloud features rotating around a typhoon center were extracted by their model from the satellite infrared videos. Moreover, models trained with long-wave infrared channels outperformed a water vapor channel-based network. The average position across the two infrared networks has a 19.3 km median error across all intensities, which equated to a 42% lower error over a baseline technique. Later, by applying the multimodal data based on typhoon track data and satellite images, Ref. [22][116] integrated the LSTM and 3D CNN model to predict typhoon trajectory. In spite of widespread RNN structures, Ref. [23][117] studied an echo state network to track the typhoon based on the meteorological dataset, yet its potential for the image-based data still needs to be explored.