PB-SVM Ensemble: A SVM Ensemble Algorithm Based on SVM

Abstract:

As one of the most popular and effective classification algorithms, Support Vector Machine (SVM) has attracted much attention in recent years. Classifiers ensemble is a research direction in machine learning and statistics, it often gives a higher classification accuracy than the single classifier. This paper proposes a new ensemble algorithm based on SVM. The proposed classification algorithm PB-SVM Ensemble consists of some SVM classifiers produced by PCAenSVM and fifty classifiers trained using Bagging, the results are combined to make the final decision on testing set using majority voting. The performance of PB-SVM Ensemble are evaluated on six datasets which are from UCI repository, Statlog or the famous research. The results of the experiment are compared with LibSVM, PCAenSVM and Bagging. PB-SVM Ensemble outperform other three algorithms in classification accuracy, and at the same time keep a higher confidence of accuracy than Bagging.

• Periodical: Applied Mechanics and Materials (Volumes 701-702)
• Pages: 58-62
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.701-702.58
• Online since: December 2014
• Authors: Bo Wang, Yu Kai Yao, Xiao Ping Wang, Xiao Yun Chen*
• Keywords: Bagging, Classifiers Ensemble, Machine Learning, PCA, SVM

References

[1] V. Vapnik, Statistical Learning Theory, Wiley, New York, (1998).

[2] Suykens J A K, Vandewalle J. Least squares support vector machine classifiers[J]. Neural processing letters, 1999, 9(3): 293-300.

[3] Chauchard F, Cogdill R, Roussel S, et al. Application of LS-SVM to non-linear phenomena in NIR spectroscopy: development of a robust and portable sensor for acidity prediction in grapes[J]. Chemometrics and Intelligent Laboratory Systems, 2004, 71(2): 141-150.

DOI: 10.1016/j.chemolab.2004.01.003

[4] Espinoza M, Suykens J A K, De Moor B. Short term chaotic time series prediction using symmetric LS-SVM regression[C]/Proc. of the 2005 International Symposium on Nonlinear Theory and Applications (NOLTA). 2005: 606-609.

[5] Adankon M M, Cheriet M. Model selection for the LS-SVM. Application to handwriting recognition[J]. Pattern Recognition, 2009, 42(12): 3264-3270.

DOI: 10.1016/j.patcog.2008.10.023

[6] Platt J. Sequential minimal optimization: A fast algorithm for training support vector machines[J]. (1998).

DOI: 10.7551/mitpress/1130.003.0016

[7] Shlens J. A tutorial on principal component analysis[J]. arXiv preprint arXiv: 1404. 1100, (2014).