基于Spark的混合协同过滤算法改进与实现.

Aiming at optimizing and improving a hybrid collaborative filtering based on Spark platform for its sparsity, scalability and personalized recommendation by using the method of algorithm integration, this paper took the model of Stacking integration to integrate multiple weak recommender units in a linearly weighted into a comprehensive recommender. Firstly, this algorithm optimized the collaborative filtering based on the nearest neighbor by presorting and adjusting the similarity calculation strategy with popularity and praise degree, and improved the rationality and complexity of similarity calculation. It solved the problem of score sparsity to some extent. At the same time, this algorithm integrated closely distributed computing platform, which could make full use of the advantages of distributed platform to design and implement an increment iterative model of recommendation algorithm by using the Spark streaming and distributed storage structure. It solved the problem that collaborative filtering algorithm was hard to expand and made poor real-time performance. The experimental data used UCI public data set named MovieLens and NetFlix films’ score. The experimental results show that the improved algorithm has a good performance and makes great progress in personalized recommendation, accuracy and scalability compared with the previous algorithms. It provides a feasible algorithm integration scheme for the application of the recommended system. [ABSTRACT FROM AUTHOR]