Research on Applied Technology in Experiments with Three Boosting Algorithms

Abstract:

Boosting algorithms are a means of building a strong ensemble classifier by aggregating a sequence of weak hypotheses. An ensemble consists of a set of independently trained classifiers whose predictions are combined when classifying novel instances. Previous research has shown that an ensemble as a whole is often more accurate than any of the single classifiers in the ensemble. In this paper we use applied technology to built an ensemble using a voting methodology of Boosting-BAN and Boosting-MultiTAN ensembles with 10 sub-classifiers in each one. We performed a comparison with Boosting-BAN and Boosting-MultiTAN ensembles with 25 sub-classifiers on standard benchmark datasets and the proposed technique was the most accurate. These results argue that boosting algorithms deserve more attention in machine learning and data mining communities.

• Periodical: Advanced Materials Research (Volume 908)
• Pages: 513-516
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.908.513
• Online since: March 2014
• Authors: Xiao Wei Sun*, Hong Bo Zhou
• Keywords: Bayesian Network Classifier, Boosting, Combination Method, Data Mining (DM)

References

[1] Schapire R E, Freund Y, Bartlett Y et al. Boosting the margin: A new explanation for the effectiveness of voting methods. In: Douglas H Fisher eds. Proc of the 14th Int' 1 Conf on Machine Learning., San Francisco: Morgan Kaufmann, 1997, 322-330.

[2] Breiman L. Bias, variance, and arcing classifiers Tech. rep. 460, Department of Statistics, University of California, Berkeley, CA. (1996).

[3] Kohavi R, Kunz C. Option decision trees with majority votes. In Proc. 14th International Conference on Machine Learning, 1997, 161-169.

[4] Bauer E, Kohavi R. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning, 1999, 36 (1/2), 105-139.

[5] Dietterich T.G. Ensemble methods in machine learning. In Kittler, J., Roli, F., eds.: Multiple Classifier Systems. Lecture Notes Computer Sciences, Vol. 1857, 2001, 1-15.

DOI: 10.1007/3-540-45014-9_1

[6] Freund Y, Robert S. Experiments with a New Boosting Algorithm. Proceedings of ICML'96, 148-156.

[7] Wu Qinghua, Liu Hanmin, Sun yuxin, Xie Fang, Zhang Jin, Yan Xuesong. Research of function optimization algorithm. TELKOMNIKA , 2012, Vol. 10, 4, 858-863.

[8] Srikanth Pullabhatla, Ashwani Kumar Chandel, Anil Naik Kanasottu. Inverse S-Transform Based Decision Tree for Power System Faults Identification. TELKOMNIKA , 2011, Vol. 9, 1, 99-106.

DOI: 10.12928/telkomnika.v9i1.674

[9] UCI Machine Learning Repository. http: /www. ics. uci. edu/~mlearn/ML Repository html.