Matrix Factorization Recommendation Algorithm Based on Attention Interaction: Comparison
View Latest Version
Please note this is a comparison between Version 2 by Camila Xu and Version 1 by chengzhi mao.

Recommender systems are widely used in e-commerce, movies, music, social media, and other fields because of their personalized recommendation functions. The recommendation algorithm is used to capture user preferences, item characteristics, and the items that users are interested in are recommended to users.

  • artificial intelligence
  • recommender system
  • collaborative filtering

1. Introduction

With the development of Internet technology, the amount of network information has increased rapidly, resulting in information overload. The problem of information overload renders users unable to obtain effective information quickly [1,2,3,4,5,6][1][2][3][4][5][6]. A recommendation system [7] has an excellent effect in addressing this problem. A recommender system is an online personalized recommendation system that helps users find and recommend items of interest by searching, and analyzing user preferences [8,9][8][9]. Currently, the main methods for recommender systems to learn user preferences fall into three categories [10,11][10][11]: content-based methods [12], collaborative filtering methods [13[13][14][15][16][17][18][19][20][21],14,15,16,17,18,19,20,21], and hybrid recommendation methods [22]. In essence, these algorithms solve the common problems of data sparsity, data scalability, and cold starts in recommender systems from different perspectives. For instance, Gulzar et al. [23] proposed a new clustering algorithm based on an ordered clustering algorithm (OCA), which aims to reduce the impact of cold starts, and data sparsity. Wu et al. [24] proposed the sampled SoftMax (SSM) loss as an efficient substitute for SoftMax loss, which can optimize long-tail recommendation.
Collaborative filtering is a popular recommendation algorithm. Based on the user behavior data (such as the ratings of users over items, user comments on items, and so on), it determines the correlation between users and items to provide personalized recommendations for users. The model-based recommendation is the most common collaborative filtering algorithm. Especially in recent years, models integrating an attention mechanism have become a research hotspot in the field of recommendation systems. The basic principle of the attention mechanism imitates the human visual system to assign different weights when processing information at different positions in the input sequence.
More and more researchers have integrated an attention mechanism to different degrees to solve the problems of data sparsity, data scalability, and cold starts.
Wang et al. [25] proposed a multi-attention deep neural network (MADNN) recommendation model based on embedding, and matrix factorization that can effectively alleviate data sparsity and cold start problems. The model enhances the interactivity of user/item embedding through a multi-attention mechanism. However, the dimensions of interactive user/item embeddings must be consistent, which may cause information loss caused by the number of features of different users, and items embedded into the same dimension.
Zhang et al. [26] proposed a probabilistic matrix factorization recommendation for self-attention mechanism convolutional neural networks with item auxiliary information, which can solve the problem of data sparsity in recommendation systems. The model adds a self-attention mechanism to the convolutional layer to establish the interaction between the auxiliary information of different channels. However, there is also the limitation that the dimensions of the auxiliary information of different channels must be consistent.

2. Matrix Factorization Recommendation Algorithm Based on Attention Interaction

Collaborative filtering algorithms based on matrix factorization (MF) are widely used, owing to their simplicity and ease of implementation. However, the rating matrix obtained by MF generally has the features of highly sparse data and uneven distribution, which lead to problems such as low recommendation performance, cold starts, and long tails [27,28][27][28]. To solve these problems, many researchers have proposed improved MF algorithms. Koren et al. [14] proved that a latent factor vector can enhance the ability of a model to deal with coefficient features. Badrul et al. [29] proposed singular value decomposition (SVD) to learn a user–item rating information matrix. However, the MF model learned using SVD was prone to overfitting. Subsequently, Funk [30] proposed the FunkSVD model, which adds a regularizer to the conventional SVD method to avoid overfitting the MF model. Koren et al. [14] Proposed a BiasSVD model with a bias term to solve the problem of large fluctuations in user-rating information. Based on BiasSVD, Koren [31,32][31][32] proposed the SVD++ model with implicit information to solve the cold-start problem caused by score sparsity. The abovementioned improved matrix factorization algorithm achieved excellent results in solving the sparsity problem of user–item rating information. Nevertheless, the simple vector dot product cannot establish a nonlinear relationship between the latent features of users and items, and the user and item features may not make full use of the latent space, leading to limited recommendation performance. Deep learning combined with matrix factorization has gradually become a mainstream research topic because of its nonlinear expression ability to establish a model. Li et al., [33] proposed a POI recommendation method fusing auxiliary attribute information based on the neural matrix factorization, integrating the convolutional neural network and attention mechanism (NueMF-CAA) to alleviate the data-sparsity problem. He et al., [34] introduced neural networks based on generalized matrix factorization (GMF). To express the nonlinear relationship between latent features, Tian et al. [35] proposed a deep matrix factorization (DMF) model that combines deep neural networks and matrix factorization techniques. DMF adds multiple hidden layers after the fully connected layers of the neural network to model higher-order interactions between users and items. Deep neural networks (DNNs) establish nonlinear relationships using weighted summation and reactivation, which may not highlight the key feature information. Therefore, the fusion of deep neural networks and attention mechanisms [36] has become a popular research topic. Wang et al. [37] proposed a convolutional neural network model based on the attention mechanism for CAPTCHA recognition, and their experimental results showed that the accuracy of CAPTCHA recognition was 93.27%. He et al. [38] proposed the inner attention-based recurrent neural network GATE function (IARNN-GATE), which uses the attention mechanism in the gate function of an RNN to control the information transmission of states between the hidden layers. Zhou et al. [39] proposed a recurrent neural network–attention mechanism model (RNN-AM) for microblog sentiment classification. Zhou et al. [40] proposed an image-denoising algorithm based on an attention mechanism and residual block, which effectively solved the problem of real image noise.

 

References

  1. Wang, M.Q.; Tang, C.L. Content supply-side information Overload Problem and Optimization Strategy: A Case study of Internet content platform. Sci-Tech Inf. Dev. Econ. 2022, 7, 30–37.
  2. Hu, Q.; Zhu, D.J.; Wu, H.L.; Wu, L.H. Survey on Intelligent recommender Systems. Comput. Syst. Appl. 2022, 31, 4758.
  3. Huang, Y.H.; Wang, W.J.; Liu, H.; Zhou, Z.K. Research Progress on Over-specialization in Personalized Information Recommendation. Inf. Sci. 2022, 40, 185–192.
  4. Zhang, S.Z.; Bai, Z.J.; Li, P.; Chang, Y.Y. Multi-Graph Convolutional Network for Fine-Grained and Personalized POI Recommendation. Electronics 2022, 11, 2966.
  5. Wang, W.; Du, Y.X.; Zheng, X.L.; Zhang, C. Neural Collaborative Recommendation Algorithm Based on Graph convolutional Self-attention Mechanism. Comput. Eng. Appl. 2023, 59, 247–258.
  6. Li, J.; Kameda, H. Load balancing problems for Multi-class jobs load balancing problems for multi-class jobs in distributed parallel computer systems. IEEE Trans. Comput. 1998, 47, 322–333.
  7. Schafer, J.B.; Konstan, J.; Riedl, J. Recommender systems in e-commerce. In Proceedings of the 1st ACM Conference on Electronic Commerce, Denver, CO, USA, 3–5 November 1999; pp. 158–166.
  8. Alberto, C.; Fabio, R. Recommender systems by means of information retrieval. In Proceedings of the International Conference on Web Intelligence, Mining and Semantics, Sogndal, Norway, 25–27 May 2011; p. 57.
  9. Francesco, R.; Lior, R.; Bracha, S. Recommender systems: Introduction and Challenges. In Recommender Systems Handbook; Springer: New York, NY, USA, 2015; pp. 1–34.
  10. Gediminas, A.; Alexander, T. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 2005, 17, 734–749.
  11. Jure, L.; Anand, R.; Jeffrey, D.U. Mining of Massive Datasets; Cambridge University Press: Cambridge, UK, 2014; pp. 307–326.
  12. Balabanovic, M.; Shoham, Y. Fab: Content-Based, collaborative recommendation. Commun. ACM 1997, 40, 66–72.
  13. Linden, G.; Smith, B.; York, J. Amazon.com recommendations: Item-to-Item collaborative filtering. IEEE Internet Comput. 2003, 7, 76–80.
  14. Koren, Y.; Bell, R.; Volinsky, C. Matrix factorization techniques for recommender systems. Computer 2009, 42, 30–38.
  15. Yang, B.C.; Li, Y.Z. Deep collaborative filtering recommendation algorithm Fusing explicit and Implicit Features. J. Liaoning Tech. Univ. Nat. Sci. 2023, 42, 354–361.
  16. Wu, W.; Xu, S.S.; Guo, S.S.; Li, X.Y. Research on point-of-interest combined recommendation algorithm based on location-based social network. Netinfo Secur. 2023, 23, 75–84.
  17. Lou, L. Research on the Implementation of College Graduate Recommendation System Based on Collaborative Filtering Algorithm. China Internet Wkly. 2023, 15, 37–39.
  18. Zhang, Y.H.; Zheng, J.Y.; Liu, L. Design and implementation of job recommendation system based on collaborative filtering algorithm. Mod. Comput. 2023, 29, 109–112.
  19. Guo, X.Y.; Shen, Y.Q.; Cui, Y. Collaborative filtering recommendation algorithm based on fuzzy clustering and user interest. Softw. Guide 2023, 22, 124–131.
  20. Gu, Y.R.; Shi, J.W.; Huang, L.Y. Neural Collaborative Filtering Recommendation Algorithm Based on Multi-head graph Attention Mechanism. J. Chin. Comput. Syst. 2023, 18, 1–9.
  21. Zhang, Q.; Yu, J.; Zhang, T.; Li, Y.F. Research on Knowledge Recommendation Model Based on Collaborative Filtering. Softw. Eng. 2023, 26, 36–39.
  22. Good, N.; Schafer, J.B.; Konstan, J.A.; Borchers, A.; Sarwar, B.; Herlocker, J.; Riedl, J. Combining collaborative filtering with personal agents for better recommendations. In Proceedings of the National Conference on Artificial Intelligence, Orlando, FL, USA, 18–22 July 1999; pp. 439–446.
  23. Gulzar, Y.; Alwan, A.A.; Abdullah, R.M.; Abualkishik, A.Z.; Oumrani, M. OCA: Ordered Clustering-Based Algorithm for E-Commerce Recommendation System. Sustainability 2023, 15, 2947.
  24. Wu, J.C.; Wang, X.; Gao, X.Y.; Chen, J.W.; Fu, H.C.; Qiu, T.Y.; He, X.N. On the Effectiveness of Sampled Softmax Loss for Item Recommendation. arXiv 2022, arXiv:2201.02327.
  25. Wang, J.; Liu, L. A multi-attention deep neural network model base on embedding and matrix factorization for recommenda-tion. Int. J. Cogn. Comput. Eng. 2020, 1, 70–77.
  26. Zhang, C.K.; Wang, C. Probabilistic Matrix Factorization Recommendation of Self-Attention Mechanism Convolutional Neural Networks with Item Auxiliary Information. IEEE Access 2020, 8, 208311–208321.
  27. Guan, F.; Zhou, Y.; Zhang, H. Research on optimization of collaborative filtering recommendation algorithm in personalized recommendation system. Oper. Res. Manag. Sci. 2022, 31, 9–14.
  28. Jin, Y.; Chen, H.M.; Luo, C. Interest Capture Recommendation Algorithm Based on Knowledge Graph. Comput. Sci. 2023, 18, 1–14.
  29. Sarwar, B.; Karypis, G.; Konstan, J.; Riedl, J. Incremental singular value decomposition algorithms for highly scalable recom-mender systems. In Proceedings of the Fifth International Conference on Computer and Information Science, Seoul, Republic of Korea, 28–29 November 2002; Volume 1, pp. 27–28.
  30. Simon, F. Funk-SVD. Available online: http://sifter.org/~simon/journal/20061211.html (accessed on 1 January 2024).
  31. Koren, Y. Factor in the neighbors: Scalable and accurate collaborative filtering. Knowl. Discov. Data 2010, 4, 1–24.
  32. Koren, Y. Collaborative filtering with temporal dynamics. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France, 28 June–1 July 2009; pp. 447–456.
  33. Li, X.Y.; Xu, S.H.; Jiang, T.; Wang, Y.; Ma, Y.; Liu, Y.M. POI Recommendation Method of Neural Matrix Factorization Integrating Auxiliary Attribute Information. Mathematics 2022, 10, 3411.
  34. He, X.N.; Liao, L.Z.; Zhang, H.W.; Nie, L.Q.; Hu, X.; Chua, T.S. Neural Collaborative Filtering. In Proceedings of the International World Wide Web Conferences Steering Committee, Perth, Australia, 3–7 May 2017.
  35. Tian, Z.; Pan, L.M.; Yin, P.; Wang, R. Deep Matrix Factorization Recommendation Algorithm. J. Softw. 2021, 32, 3917–3928.
  36. Vaswani, A.; Shazeer, N.M.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, L.; Polosukhin, I. Attention is All you Need. Neural Inf. Process. Syst. 2017, 30.
  37. Wang, Z.A.; Li, Y.L.; Shang, Z.Y.; Li, G.B. Attention-based Convolutional Neural Networks for CAPTCHA Recognition. J. Southwest Minzu Univ. Nat. Sci. Ed. 2023, 49, 303–311.
  38. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778.
  39. Zhou, X.Z.; Li, S.; Sui, D. Microblog Sentiment Analysis Based on Deep Learning and Attention Mechanism. J. Nanjing Norm. Univ. Nat. Sci. Ed. 2023, 46, 115–121.
  40. Zhou, L.M.; Zhou, D.M. Real image denoising based on attention mechanism and residual block. Comput. Eng. Des. 2023, 44, 1451–1458.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/)
Video Production Service
Feedback