Online Learning Algorithm of Gaussian Process Based on Adaptive Natural Gradient for Regression


Article Preview

Gaussian Process (GP) is a new learning method on nonlinear system modeling. The most common way of model training is conjugate gradient method, but this method should compute Heisenberg matrix which needs much computing resource. It is not a suitable training method for online learning algorithm. There is one online learning algorithm of GP which is named sparse online GP now. This algorithm has constraint to the training data sets. In order to satisfy the real-time modeling without the limit of the training data sets, an online algorithm of GP based on adaptive natural gradient (ANG) is proposed in this paper. The algorithm is applied in Continuous Stirred Tank Reactor (CSTR) modeling and the sparse online GP is also applied in CSTR modeling for comparison. Obtained from the simulation results, the algorithm is effective and has higher Accuracy compared with the sparse online GP algorithm.



Advanced Materials Research (Volumes 139-141)

Edited by:

Liangchi Zhang, Chunliang Zhang and Tielin Shi




Q. Q. Shen and Z. H. Sun, "Online Learning Algorithm of Gaussian Process Based on Adaptive Natural Gradient for Regression", Advanced Materials Research, Vols. 139-141, pp. 1847-1851, 2010

Online since:

October 2010




[1] Information on http: /www. inference. phy. cam. ac. uk/mackay/BayesGP. html.

[2] M. Seeger: International Journal of Neural System, Vol. 14 (2004) No. 2, pp.69-106.

[3] H.Z. Wang: Chemical Engineering, Vol. 58(2007) No. 11, pp.2840-2845 (In Chinese).

[4] M.P. Deisenroth, J. Peters and C.E. Rasmussen: Proc. American Control Conference (Seattle, Washington, USA, June 11 - 13, 2008). Vol. 1, pp.4480-4485.

[5] R.M. Neal: Bayesian Learning for Neural Networks (Ph.D., Dept. Comput. Sci., Univ. Toronto, Canada, 1995).

[6] C.W. Neural, C.K.I. Williams and C.E. Rasmussen: In Advances in Neural Information Processing Systems 8 (MIT Press, Cambridge, MA 1996), pp.514-520.

[7] S.I. Amari and S.C. Douglas: Acoustics, Speech and Signal Processing, vol. 2 (1998) No. 5, pp.1213-1216.

[8] L. Csato and M. Opper: Neural Computation, Vol. 14 (2002) No. 3, pp.641-668.

[9] H. Park, S.I. Amari and K. Fukumizu: Neural Networks, Vol. 13 (2000) No. 7, pp.755-764.