Application and Research on Distributed Collaborative Filtering Recommendation Algorithm Based on Hadoop

Abstract:

Coupled with exponential expansion of the data, efficient computing of existing recommendation algorithm has become an important issue, and the traditional collaborative filtering recommendation algorithm also exist the problem of sparsity. Based on the detailed analysis, the article introduce Hadoop platform into improved collaborative filtering recommendation algorithm, the improved collaborative filtering recommendation algorithm solve the problem of data sparsity, MapReduce parallel computing of recommendation also solve the promble of computational efficiency. In the experiments, the comparative analysis between Hadoop platform implementation and the previous implementation draws the conclusion that the Hadoop platform improves collaborative filtering recommendation algorithm computation efficiently under conditions of large data sets.

• Periodical: Applied Mechanics and Materials (Volumes 713-715)
• Pages: 1615-1621
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.713-715.1615
• Online since: January 2015
• Authors: Xiu Juan Li*, He Biao Yang
• Keywords: Cloud Computing, Collaborative Filtering, Hadoop, MapReduce

References

[1] G ADOMAVICIUS, A TUZHILIN. IEEE Trans on Knowledge and Data Engineering, 2005, 17(6): 734-749.

[2] B SARWAR, G KARYPIS, J KONSTAN, et al. Proceedings of the 10th International Conference on world Wide Web. New York: ACM Press. 2001: 285-295.

[3] G.R. Mettam, L.B. Adams, B.S. Jones, R.Z. Smith (Eds. ), Introduction to the Electronic Age, E-Publishing Inc., New York, 1999, pp.281-304.

[4] AHN H J, Information Sciences, 2008, 178(1): 37-51.

[5] Ailin Deng, Yangyong Zhu, Bole Shi. Journal of Software, 2003, 14(9): 1621-1628.

[6] Qiao Li, Xiao Zheng. Computer Science, 2011 38(04): 32-37.

[7] Zhiyun Zheng, Buyuan Li, Lun Li. Computer Science, 2013, 12(40): 259-263.

[8] JEFFREY D, SANJAY G. Proceedings of the Sixth Symposium on Operating Systems Design and Implementation. 2004: 137-149.

[9] Grossman M, Breternitz Jr M, Sarkar V: MapReduce on Distributed Heterogeneous Platforms through Seamless Integration of Hadoop and OpenCL[C]/IPDPS Workshops. 2013: 1918-(1927).

DOI: 10.1109/ipdpsw.2013.246

[10] Yuan Zhou. Research on Recommendation Algorithm Based on Cloud Computing. Sichuan: (2012).

[11] BREESE J, HECHERMAN D, KADIE C. Proc of the 14th Conference on Uncertainty in Artificial Intelligence. 1998: 43-52.

[12] QiangXiao, Qinghua Zhu, Hua Zheng. New Technology of Library and Information Service, 2013, 229(1): 83-88.