Study of Multiple-Kernel Relevance Vector Machine Based on Kernel Alignment

Abstract:

The relevance vector machine (RVM) was a Bayesian framework for learning sparse regression models and classifiers, it used single kernel function to map training data from low dimension sample space to high dimension feature space. The prediction accuracy and generalization of traditional single-kernel RVM (sRVM) were not ideal both in classification and regression, so we constructed homogeneous and heterogeneous multiple kernels function (MKF) by kernel function combination in which we testified the validity of basic kernel function (BKF) and its parameters we employed by kernel alignment (KA), then we acquired optimized multiple-kernel RVM (mRVM). Experiment results on LIBSVM datasets not only indicate that both homogeneous and heterogeneous multiple-kernel RVM we constructed possess lower error rate in classification and smaller root mean square (RMS) in regression than single-kernel RVM, but also prove the effectiveness of kernel alignment.

• Periodical: Applied Mechanics and Materials (Volumes 239-240)
• Pages: 1308-1312
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.239-240.1308
• Online since: December 2012
• Authors: Lv Zhao, Yi Dan Su, Hua Qin, Pian Pian Ma
• Keywords: Heterogeneous, Homogeneous, Kernel Alignment, Kernel Function, Multiple Kernel, Relevance Vector Machine

References

[1] M.E. Tipping: J. Machine Learning Research Vol. 1 (2001), p.211.

[2] V.N. Vapnik: Statistical Learning Theory (Wiley, New York 1998).

[3] B. Schoelkopf, A.J. Smola and K.R. Muller: Neural Computation Vol. 10 (1998), p.1299.

[4] B. Schoelkopf, S. Mika, C.J.C. Burges, P. Knirsch, K.R. Muller and G. Ratsch: IEEE Trans. on Neural Network Vol. 10 (1999), p.1000.

DOI: 10.1109/72.788641

[5] K.R. Muller,S. Mika, G. Ratsch, K. Tsuda and B. Scholkopf: IEEE Trans. on Neural Network Vol. 12 (2001), p.181.

DOI: 10.1109/72.914517

[6] B. Scholkopf and A.J. Smola: Learning with Kernels (The MIT Press, Massachusetts 2002).

[7] T.J. Shawe and N. Cristianini N: Kernel Methods for Pattern Analysis (Cambridge University Press, Cambridge 2004).

[8] T. Damoulas and M.A. Girolami: Bioinformatics Vol. 24 (2008), p.1264.

[9] F.R. Bach: J. Machine Learning Research Vol. 9 (2008), p.1179.

[10] S. Sonnenburg, G. Ratsch, C. Schafer and B. Scholkopf: J. Machine Learning Research Vol. 7 (2006), p.1531.

[11] H.A. Ben and W.S. Noble: Bioinformatics Vol. 21 (2005), p.38.

[12] A. Rakotomamonjy, F.R. Bach, S. Canu and Y. Grandvalet: Proc. 24th Int. Conf. on Machine Learning (ACM, Oregon 2007), p.775.

DOI: 10.1145/1273496.1273594

[13] A. Rakotomamonjy, F.R. Bach, S. Canu and Y. Grandvalet: J. Machine Learning Research Vol. 9 (2008), p.2491.

[14] D.N. Zheng, J.X. Wang and Y.N. Zhao: Neurocomputing Vol. 70 (2006), p.420.

[15] Z. Yang, J. Guo, W. Xu, X.F. Nie, J. Wang and J.J. Lei: Proc. 3rd Int. Symp. Conf. on Neural Networks (Springer, China 2006), p.1030.

[16] G.R.G. Lanckriet, N. Cristianini and P. Bartlett: J. Machine Learning Research Vol. 5 (2004), pp.27-72.