A Novel Smooth Support Vector Regression Based on CHKS Function

Abstract:

This paper presents a new smooth approach to solve support vector regression (SVR). Based on Karush-Kuhn-Tucker complementary condition in optimization theory, a smooth unconstrained optimization model for SVR is built. Since the objective function of the unconstrained SVR model is non-smooth, we apply the smooth techniques and replace the ε-insensitive loss function by CHKS function. Newton-Armijo algorithm is used to solve the smooth CHKS-SSVR model. Primary numerical results illustrate that our proposed approach improves the regression performance and the learning efficiency.

• Periodical: Applied Mechanics and Materials (Volumes 44-47)
• Pages: 3746-3751
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.44-47.3746
• Online since: December 2010
• Authors: Qing Wu
• Keywords: CHKS Function, Newton-Armijo Algorithm, Optimization Theory, Smooth Approximation, Support Vector Regression (SVR)

References

[1] Y. Y. ZHAO, A. N. CAI. Fingerprint Image Segmentation Based on Support Vector Machines, Journal of Beijing University of Posts and Telecommunications(Chinese), Vol. 29(2006), pp.38-41.

[2] H. Yang, L. Chan, and I. King. Support Vector Machine Regression for Volatile Stock Market Prediction, IDEAL 2002, LNCS 24412(2002), Berlin: Springer-Verlag, pp.391-396.

DOI: 10.1007/3-540-45675-9_58

[3] B. J. Chen, M. W. Chang, and C. J. Lin. Load Forecasting Using Support Vector Machines: A Study on EUNITE Competition 2001, report for EUNITE competition for Smart Adaptive System(2001). Available: http: /www. eunite. org.

DOI: 10.1109/tpwrs.2004.835679

[4] D. Matterra and S. Haykin. Support Vector Machines for Dynamic Reconstruction of a Chaotic System, in Advances in Kernel Methods, B. Schölkopf, C.J.C. Burges, and A.J. Smola Eds., MIT Press(1999), pp.211-241.

DOI: 10.7551/mitpress/1130.003.0018

[5] A. Ding, X. Zhao, and L Jiao. Traffic Flow Time Series Prediction Based On Statistics Learning Theory, , the IEEE 5th International Conference on Intelligent Transportation Systems, Proceedings( 2002), pp.727-730.

DOI: 10.1109/itsc.2002.1041308

[6] Y.J. Lee, W.F. Hsieh and C.M. Huang. ε-SSVR: A Smooth Support Vector Machine for ε-Insensitive Regression, IEEE Transactions on Knowledge and Data Engineering, Vol. 17 (2005), pp.678-685.

DOI: 10.1109/tkde.2005.77

[7] L. Armijo. Minimization of functions having Lipschitz-continuous first partial derivatives, Pacific Journal of Mathematics, Vol. 16 (1966), pp.1-3.

DOI: 10.2140/pjm.1966.16.1

[8] D. P. Bertsekas. Nonlinear programming,. Athena Scientific, Belmont, MA, second edition(1999).

[9] K. Hotta, M. Inaba and A. Yoshise. A complexity analysis of a smoothing method using CHKS-functions for monotone linear complementarity problems, Computational Optimization and Applications, Vol. 17 (2000), pp.183-201.

[10] S. H. Pan, X. S. Li. An efficient algorithm for the smallest enclosing ball problem in high dimensions, Applied Mathematics and Computation, Vol. 172 (2006), p.49–61.

DOI: 10.1016/j.amc.2005.01.127

[11] P. M. Murphy and D. W. Aha. UCI machine learning repository, http: /www. ics. uci. edu/~mlearn/MLReposito-ry. html(1996).

[12] DELVE. Data for evaluating learning in valid experiments, Kin-family dataset, , http: /www. cs. toronto. edu/~delve/data/kin/desc. html(1996).