The task of sequence-based recommendation is to predict the next item the user will buy based on the user’s behavioral sequence. The Markov chain-based model is a classic sequence-based recommendation model [9,10]. Because of the development of deep learning, RNN-based models were introduced into sequence-based recommendations. Bal et al. used the RNN-based model GRU4Rec [11] for the first time to predict the next item that the user might be interested in. However, because Markov assumes that the current interaction only depends on one or several recent interactions, the results predicted by models would only be dependent on the user’s most recent behavioral sequence. In addition, CNN-based models were also introduced into sequential recommendations, such as Caser [12]. Since CNN-based models do not have a strong ordering assumption for the interaction in the sequence, the CNN-based recommendations can make up for the disadvantage based on RNN to a certain extent. Transformer [13], as a sequence-based model, achieved state-of-the-art performance and efficiency for machine translation tasks because of ’self-attention’. Therefore, an attention mechanism was introduced into sequence-based recommendations [14], and Wang et al. [15] proposed SASRec, which focuses more on the whole sequence instead of the most recent behavioral sequence. In addition, Li et al. [16] proposed TiSASRec, which is based on SASRec and uses the temporal factor, and Sun et al. [7] proposed Bert4Rec using Bidirectional Encoder Representations from Transformers (BERT). Yukuo et al. [17] introduced multiple interests into sequence-based recommendations to improve the model’s performance. Although the above models are efficient, they require a fixed length, which means that if the user’s sequence exceeds this length these models will not be able to express the user’s interests fully. In order to solve this problem, Bai et al. proposed LSDA [18], which uses multiple LSTM to learn the whole sequences of users, and Bo et al. proposed HAM [19]; this model uses MLP to learn the whole sequences of users. Nevertheless, these models only pay attention to the sequence and do not pay attention to the spatial signals of data.

Initially, traditional matrix factorization models [20,21] were used for the recommendation system. With the rise in deep learning methods, traditional matrix factorization models have been gradually replaced by deep neural networks, especially graph neural networks (GNN). Berg et al. [22] utilized a graph convolutional neural network in a recommendation system, and then Wang et al. [23] improved it and proposed NGCF. However, these models use the Laplacian matrix, and the matrix operation is troublesome; hence, He et al. [24] simplified the matrix operation and proposed LightGCN, which improved the efficiency of the operation in sparse datasets. Fan et al. [25] improved LightGCN by changing the global graph into a subgraph to improve the generalization of the model. In addition, Liang et al. [26] used auto-encoders for recommendations, which can effectively provide feedback on the implicit data of users. At the same time, Wang et al. [27] combined the attention mechanism with the graph neural network, proposed GAT, and applied it to recommendation systems [28]. Some scholars also use dynamic graphs to improve the recommendation ability of models [29,30,31,32]. GCN-based models can fully use the user’s behaviors and can also use the spatial signal of the data to learn the user’s holistic interests. However, GCN-based models cannot utilize the user’s temporal signals and capture the recent interests effectively.

Since graph convolutional neural networks can handle graph-structured data well, some scholars combine GNN with other models and apply them to sequence-based recommendation systems. For example, SR-GNN [33], MA-GNN [34], and DGCN [35] combine GNN and GRU, which improves the generalization ability of the model. APGNN [36] combines GNN and attention mechanisms. However, these models that use GRU will focus too much on the recent behavior of users and ignore their complete behavior, and GRU’s computational complexity is high. Some scholars have also introduced a memory network into the recommendation system; thereby, the matrix can more explicitly and dynamically store and update the historical interactions in the sequence to improve the expressive ability of the model. For example, Chen et al. [37] and Huang et al. [38] added a memory network to the GRU to improve the expression ability of the model. Yuan et al. [39] used a memory network for session-based recommendations, and Wang et al. [40] used an attention network for the next location recommendation. Tan et al. [41] and Hsu et al. [42] used an attention graph for sequential recommendations. Meanwhile, some scholars introduce models that pay more attention to sequence. Zhu et al. proposed GES [43], combining GNN and Transformer, but this model focuses too much on the user’s recent behavioral sequence. These hybrid models combine GCN-based models and sequence-based models, but the above models still focus on the recent behavior sequence of users and lack attention to the complete behavior of users.