A Multi-Classification Method Based on Gaussian Processes

Abstract:

Gaussian processes (GPs) is a very promising technology that has been applied both in the regression problem and the classification problem. In recent years, models based on Gaussian process priors have attracted much attention in the machine learning. Binary (or two-class, C=2) classification using Gaussian process is a very well-developed method. In this paper, a Multi-classification (C>2) method is illustrated, which is based on Binary GPs classification. A good accuracy can be obtained through this method. Meanwhile, a comparison about decision time and accuracy between this method and Support Vector Machine (SVM) is made during the experiments.

• Periodical: Applied Mechanics and Materials (Volumes 198-199)
• Pages: 1333-1337
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.198-199.1333
• Online since: September 2012
• Authors: San Xi Wei, Zong Hai Sun
• Keywords: Gaussian Processes, Multi-Classification, Support Vector Machine (SVM)

References

[1] C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning, The MIT press, Cambridge (2006).

[2] R. M. Neal, Regression and classification using Gaussian process priors. In J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, editors, Bayesian Statistics 6, pages 475–501. Oxford University Press (1998).

[3] Lehel Csato, Gaussian Processes - Iterative Sparse Approximations, ASTON UNIVERSITY March (2002).

[4] Y. Bazi, and F. Melgani, Gaussian Process Approach to Remote Sensing Image Classification, IEEE transact- tions on geoscience and remote sensing, vol. 48, no. 1, January (2010).

DOI: 10.1109/tgrs.2009.2023983

[5] J. Kocijan, R. Murray-Smith, C. E. Rasmussen, A. Girard, Gaussian Process Model Based Predictive Control, Proceeding of the 2004 American Control Conference Boston, Massachuselts June 30 -July 2 (2004).

DOI: 10.23919/acc.2004.1383790

[6] R. Duda, S. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York: Wiley (2001).

[7] J. S. Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge, U.K.: Cambridge Univ. Press (2004).

[8] Bazi, Y. Melgani, F. Coll. of Eng., Al-Jouf Univ., Al-Jouf, Saudi Arabia, Gaussian Process Approach to Remote Sensing Image Classification, IEEE transactions on geoscience and remote sensing, vol. 48, no. 1, January (2010).

DOI: 10.1109/tgrs.2009.2023983

[9] R. M. Neal, Probabilistic inference using Markov chain Monte Carlo methods. Technical Report CRG-TR-93-1, Department of Computer Science, University of Toronto, (1993).

[10] F. Takahashi and S. Abe, Decision-tree-based Multi- class Support Vector Machines, Proceedings of the Ninth International Conference on Neural Information Processing, (2002).

DOI: 10.1109/iconip.2002.1202854

[11] J. Weston and C. Watkins, MultiClass Support Vector Machines. In M. Verleysen, editor Proceedings of ESA NN99, Brussels (1999).

[12] O'Hagan, Curve fitting and optimal design for prediction. Journal of the Royal Statistical Society, Series B, 40(1): 1–42 (1978).

[13] M. Kuss and C. E. Rasmussen, Assessing Approximate Inference for Binary Gaussian Process Classification, Journal of Machine Learning Research 6 (2005).

[14] Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library for support vector machines, 2001. Software available at http: /www. csie. ntu. edu. tw/~cjlin/libsvm.

[15] Danial Keysers, [Online]. http: /www-i6. informatik. rwth-aachen. de/~keysers/usps. html.

[16] C. E. Rasmussen & H. Nickisch, http: /www. gaussian process. org/gpml/code/ matlab /doc/. (2010).

[17] S. Canu, S. Canu  and Y. Grandvalet and V. Guigue and A. Rakotomamonjy, SVM and Kernel Methods Matlab. Toolbox, Perception Systemes et Information, INSA de Rouen, Rouen, France, (2005).

[18] V. Vapnik, Statistical Learning Theory. John Wiley & Sons, Inc., (1998).