Constraint Projection Adaptive Natural Gradient Online Algorithm for SVM

Abstract:

The training of Support Vector Machine (SVM) is an optimization problem of quadratic programming which can not be applied to the online training in real time applications or time-variant data source. The online algorithms proposed by other researchers have high computational complexity and slow training speed, which can not be well applied to the time-variant problems as well. In this paper the projection gradient and adaptive natural gradient is combined. The constraint projection adaptive natural gradient online algorithm for SVM is proposed. The computation complexity of the constraint projection adaptive natural gradient algorithm is . The learning performance is compared via prediction of the concentration of component A of Continuous Stirred Tank Reactor. The results of simulation demonstrate that the time taken by the constraint projection adaptive natural gradient online algorithm for SVM is far less than that of incremental algorithm, while keep higher prediction precision.

• Periodical: Advanced Materials Research (Volumes 139-141)
• Pages: 1692-1696
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.139-141.1692
• Online since: October 2010
• Authors: Zong Hai Sun
• Keywords: Natural Gradient, Online Algorithm, Projection, Support Vector Machine (SVM)

References

[1] V. Vapnik: Statistical Learning Theory. John Wiley and Sons, New York, (1998).

[2] V. Vapnik et al: Advances in Neural Information Processing Systems, (1997), pp.281-287.

[3] L. Byunghwa et al: Proc. SICE-ICASE International Joint Conference (2006), pp.5614-5618.

[4] G. Cauwenberghs, T. Poggio: Machine Learning, Vol. 44(2001), pp.409-415.

[5] J. H. Ma et al: Neural Computation, Vol. 15 (2003) No. 11, pp.2683-2703.

[6] Information on http: /onlinesvr. altervista. org, June (2007).

[7] D. C. Wang and B. Jiang: Control and Decision, Vol. 22(2007), pp.132-137.

[8] S. Amari: Neural Computation, (1998), pp.251-276.

[9] S. Amari, H. Park, and K. Fukumizu: Neural Computation, Vol. 12(2000), pp.1399-1409.

[10] H. P. Yin: Research on Online Algorithm of SVM and Application in Modeling of CSTR. (MS., South China University of Technology, China 2009).

[11] P. Wolfe: Quarterly of Applied Mathematics, Vol. 19(1961), pp.239-244.

[12] Shah L. U. et al., Proc. the 17 th European Signal Processing Conference (Glasgow, Scotland August 24-28, 2009), pp.884-888.