Adaptive Ensemble Learning Recommendation Algorithm

Abstract:

In order to solve the problem of low accuracy of single model collaborative filtering, this paper proposes an adaptive ensemble learning recommendation algorithm (AdaELRA). In the algorithm we minimize the prediction error through the gradient descent method, and set as the deviation coefficient to adaptive the prediction error, so that the sample weights will be more in line with the predictions. Theoretical analysis and experimental results show that the algorithm this paper proposed reduce the time complexity and it can improve the accuracy significantly.

• Periodical: Advanced Materials Research (Volumes 1044-1045)
• Pages: 1433-1436
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.1044-1045.1433
• Online since: October 2014
• Authors: Zhong Hua Luo*, Yang Yang
• Keywords: Boosting Algorithm, Difference, Ensemble Learning, Learner, Recommendation Algorithm

References

[8] proposed an ensemble learning approach based on the Boosting ensemble learning of Ada-Boost, in this method the author ordered some results produced by the weak method to make the final result. Marco Tiemann and Steffen Pauws [9] based on the decision rules proposed an ensemble learning approach, and it was used in the music recommendation system and gains a better performance than the content-based collaborative filtering. Michael Jahrer [10] proposed a heterogeneous ensemble learning method which contains SVD, neighbor, restricted Boltzmann machine, asymmetric factor model and other 19 kind models. The author use a residual training method combined all the sub-models, experimental results show that the prediction accuracy is better than any single model. Adaptive Ensemble learning Recommendation Algorithm Design base class learning algorithm. In ensemble learning we need to determine the base class learning method first to prepare for the subsequent training sample. In this paper, we use a simple and effective regression algorithm to do this job, through minimize the error to obtain assessment scores. represent the user set and is its number; represent the item set and is the number. is the rate of user for item , is the rates produced by the recommendation system, is difference, and. The users are very little who rate on every project, so the effect recommendation is difficult, and the final rate set of prediction is . If presents the weights of the impact of user v on v, presents the initial prediction score of user u for project i( is the same), presents the neighbor set of user u, represents the projects set which rated by user v. We define stand for the prediction score of user u for project i, so we can calculate use the following regression formula. (1) Where . is user u's neighbor set. We use Pearson coefficient to measure the correlation of the users to verify the effectiveness of ensemble learning algorithm. If stands for the mean of all non-empty ratings, is the mean of user u's ratings, is the average score of project i, so the prediction score of user u for project i can be initialized as follow. (2) We define indicator function as: So the total error can be expressed as formula (3) (3) In this paper we update according the gradient descent, that is calculate. Assume that effect of user u on v does not equal the effect of user v on u, so gradient can be expressed as formula (4). (4) Substituting into formula (1), we can get the predictive regression formula of the users, and use it as the weak base class algorithm. So, the base class learning algorithm proposed this paper is shown as figure 1. Algorithm 1 base class learning algorithm Input：rating matrix of user for project Output：the prediction score.

[1] calculate.

[2] Initialization according to formula (1).

[3] for i = 1 to k do.

[4] calculate.

[5] update according to.

[6] calculate according to.

[7] end for.

[8] return the prediction score of the users. Figure1 Base class learning algorithm Design the adaptive Ensemble learning Recommendation Algorithm. The adaptive ensemble learning recommendation algorithm (AdaELRA) this paper proposed, which based on the AdaBoost. RT, use coefficient of variation to replace the threshold of relative error. Calculate the mean and standard deviation of the prediction error at each step in iterative process of the ensemble learning. We should increase the weight of the sample and strengthen the training, when the difference between the prediction error of the sample and the value of multiplied by the average value. There are two advantages after the parameter was improved: 1) the coefficient of variation can be adaptively adjusted according to the situation of the prediction error, so that is more in line with forecasts; 2) we can use statistical properties of prediction error rather than a default fixed threshold to calculating , and improved the flexibility and availability. 2) In calculating the parameters, use statistical properties of prediction error, rather than to default to a fixed threshold, the algorithm improves the flexibility and availability. The algorithm process is shown as figure 2 Algorithm 2 Adaptive Ensemble Learning Recommendation Algorithm Input：1) Triples of user-project 2) The prediction score of the user 3) Iterations T of ensemble learning 4) Error factor (default value is 1) Output：Predicted user rating.

[1] initialize the iterations t=1.

[2] set user rating weight according to the distribution of mean.

[3] if t<T.

[4] predict according to formula (1).

[5] calculate the mean and standard deviation of prediction error.

[6] calculate.

[7] update the weight.

[8] t = t + 1.

[9] Weighted the entire model.

[10] Return the predicted rating Figure 2 AdaELRA Algorithm is a normalized factor, which make the summation of is 1. The time complexity of Adaboost algorithm is , where is the iterations, is the dimensions of spatial, is the category of the sample. But the time complexity of AdaELRA this paper proposed is, is better than Adaboost obviously. Experimental verification This section designed experiments to verify the performance and the accuracy of the algorithm. In the experiment we use MovieLens's dataset, which contains 3900 films rating data of 6040 users, 100 users rating data was selected as the verification dataset. The 70% of the data is the training set and the other is the test set. The hardware environment of the experiment is Intel (R) Pentium E2200 2. 19GHz quad-core CPU and 2GB of memory, the software environment is Windows 7 pro operating system and all the codes are implemented with Java 7 and Matlab2008. In the algorithm we set neighbors users as 40, and the iterations of ensemble learning as 14, and we use the mean absolute error (MAE) to evaluate the model, in the experiment we juxtaposed the performance of the single collaborative filtering recommendation model, standard AdaBoost. RT ensemble learning model and the AdaELRA model, and the contrast result is shown as figure 3. Figure 3 The contrast of different models According to figure 3 we can see that the performance is enhanced after iteration. Compare to the traditional algorithms and AdaBoost. RT, the AdaELRA this paper proposed is better. The proposed model can reduce the prediction error effectively. Conclusion This paper proposed an AdaELRA algorithm to solve the low accuracy problem of recommendation system. In the algorithm we through minimize the prediction error through gradient descent, and the coefficient of variation can adaptively according to the prediction error. The algorithm reduced the time complexity and gets a better performance than AdaBoot. RT. Experimental result shown that AdaELRA this paper proposed can improve the performance significantly. Reference.

[1] Xiaowei Xu, Fudong Wang. Trust -Based Collaborative Filtering Algorithm [C]. Proceedings of the 2012 Fifth International Symposium on Computational Intelligence and Design. NW Washington, DC USA, 2012: 321-324.

DOI: 10.1109/iscid.2012.88

[2] Ma Zhanyu, Leijon Arne. A model-based collaborative filtering method for bounded support data [C]. Proceedings - 2012 3rd IEEE International Conference on Network Infrastructure and Digital Content, IC-NIDC 2012, pp.545-548.

DOI: 10.1109/icnidc.2012.6418813

[3] Kwon Ohbyung, Jung Dongyoung. An association model based reasoning method for individualized service recommender [J]. EXPERT SYSTEMS, 2013, 30(1): 54-65.

DOI: 10.1111/j.1468-0394.2012.00621.x

[4] Wang Hai-yan, Zhang Da-yin. A Trustworthy Service Selection Model Based on Collaborative Filtering [J]. Journal of Electronics & Information Technology, 2013, 35(2): 349-354.

DOI: 10.3724/sp.j.1146.2012.00946

[5] T.G. Dietterich. Ensemble Methods in Machine Learning. In Multiple Classier Systems, Cagliari, Italy, (2000).

[6] Thomas G. Dietterich. Ensemble learning. In The Handbook of Brain Theory and Neural Networks, Second Edition, (2002).

[7] Y. Koren R. M. Bell and C. Volinsky. The bellkor solution to the netflix prize, (2007).

[8] Yoav Freund, Raj Iyer, Robert E. Schapire, and Yoram Singer. An efficient boosting algorithm for combining preferences [J]. Journal of Machine Learning Research, 4: 933–969, (2003).

[9] Tiemann M. and Pauws S. Towards ensemble learning for hybrid music recommendation. In Proceedings of RecSys. 177-178, (2007).

[10] Michael Jahrer, Andreas Töscher. Combining Predictions for Accurate Recommender systems. In proceeding of 16th Intenational Conference on Knowledge Discovery and Data Mining. (2012).

DOI: 10.1145/1835804.1835893