Unsupervised Kernel Learning Vector Quantization

Abstract:

In this paper, we propose an unsupervised kernel learning vector quantization (UKLVQ) algorithm that combines the concepts of the kernel method and traditional unsupervised learning vector quantization (ULVQ). We first use the definition of the shadow kernel to give a general representation of the UKLVQ method and then easily implement the UKLVQ algorithm with a well-defined objective function in which traditional unsupervised learning vector quantization (ULVQ) becomes a special case of UKLVQ. We also analyze the robustness of our proposed learning algorithm by means of a sensitivity curve. In our simulations, the UKLVQ with Gaussian kernel has a bounded sensitivity curve and is thus robust to noise. The robustness and accuracy of the proposed UKLVQ algorithm is also demonstrated via numerical examples.

• Periodical: Advanced Engineering Forum (Volumes 6-7)
• Pages: 243-249
• DOI: https://doi.org/10.4028/www.scientific.net/AEF.6-7.243
• Online since: September 2012
• Authors: Kuo Lung Wu
• Keywords: Kernel Method, Learning Vector Quantization, Robustness

References

[1] S.P. Lloyd, Least squares quantization in pcm, IEEE Trans. Inf. Theory 28 (2) (1982) 129-137.

DOI: 10.1109/tit.1982.1056489

[2] D. Pollard, Quantization and the method of k-means, IEEE Trans. Inf. Theory 28 (2) (1982) 199-205.

[3] C. Chinrungrueng, C.H. Séquin, Optimal adaptive k -means algorithm with dynamic adjustment of learning rate, IEEE Trans. Neural Network 6 (1995) 157-169.

DOI: 10.1109/72.363440

[4] T. Kohonen, Self-organized formation of topologically correct feature maps, Biol. Cybern. 43 (1982) 59-69.

DOI: 10.1007/bf00337288

[5] T. Kohonen, The self-organizing map, Proc. IEEE 78 (1990) 1464–1480.

[6] E. de Bodt, M. Cottrell, P. Letremy, M. Verleysen, On the use of self-organizing maps to accelerate vector quantization, Neurocomputing 56 (2004) 187-203.

DOI: 10.1016/j.neucom.2003.09.009

[7] M.M. Campos, G.A. Carpenter, S-TREE: self-organizing trees for data clustering and online vector quantization, Neural Networks 14 (2001) 505-525.

DOI: 10.1016/s0893-6080(01)00020-x

[8] J.A. Lee, M. Verleysen, Self-organizing maps with recursive neighborhood adaptation, Neural Networks 15 (2002) 993-1003.

DOI: 10.1016/s0893-6080(02)00073-4

[9] E. Yair, K. Zeger, A. Gersho, Competitive learning and soft competition for vector quantizer design, IEEE Trans. Signal Process 40 (1992) 294-309.

DOI: 10.1109/78.124940

[10] H. Ritter, K. Schulten, On the stationary state of Kohonen's self-organizing sensory mapping, Biol. Cybern. 54 (1986) 99-106.

DOI: 10.1007/bf00320480

[11] Z.P. Lo, B. Bavarian, On the rate of convergence in topology preserving neural networks, Biol. Cybern. 65 (1991) 55-63.

DOI: 10.1007/bf00197290

[12] D.Q. Zhang, S.C. Chen, A novel kernelized fuzzy c-means algorithm with application in medical image segmentation, Artificial Intelligence in Medicine. 32 (2004) 37-50.

DOI: 10.1016/j.artmed.2004.01.012

[13] D.W. Kim, K.Y. Lee, D. Lee and K.H. Lee, A kernel-based subtractive clustering method, Pattern Recognition Letters 26 (2005) pp.879-891.

DOI: 10.1016/j.patrec.2004.10.001

[14] E.A. Zanaty, S. Aljahdali and N. Debnath, A kernelized fuzzy c-means algorithm for automatic magnetic resonance image segmentation, Journal of Computational Methods in Science and Engineering 9 Supplement 2 (2009) 123-136.

DOI: 10.3233/jcm-2009-0241

[15] M. Girolami, Mercer kernel based clustering in feature space, IEEE Trans. Neural Networks 13 (2002) 780-784.

DOI: 10.1109/tnn.2002.1000150

[16] A. Likas, N. Vlassis and J.J. Verbeek, The global k-means clustering algorithm, Pattern Recognition 36 (2003) 451-461.

DOI: 10.1016/s0031-3203(02)00060-2

[17] B. Scholkopf, A. Smola and K. -R. Muller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Computation, Neural Computation 10 (1998) pp.1299-1319.

DOI: 10.1162/089976698300017467

[18] I.S. Dhillon, Y. Guan and B. Kulis, Weighted graph cuts without eigenvectors: a multilevel approach, IEEE Trans. Pattern Analysis and Machine Intelligence 29 (2007) 1944-(1957).

DOI: 10.1109/tpami.2007.1115

[19] D.W. Kim, K. Y Lee, D. Lee and K.H. Lee, Evaluation of performance of clustering algorithms in kernel-induced feature space, Pattern Recognition 38 (2005) 607-611.

DOI: 10.1016/j.patcog.2004.09.006

[20] J. Shi and J. Malik, Normalized cuts and image segmentation, IEEE Trans. Pattern Analysis and Machine Intelligence 22 (2002) 888-905.

DOI: 10.1109/34.868688