Vector Quantization by Minimizing Kullback-Leibler Divergence between the Class Label Distributions over Quantization Input and Output

Abstract:

This paper proposes a new method for vector quantization by minimizing the Divergence of Kullback-Leibler between the class label distributions over the quantization inputs, which are original vectors, and the output, which is the quantization subsets of the vector set. In this way, the vector quantization output can keep as much information of the class label as possible. An objective function is constructed and we developed an iterative algorithm to minimize it as well. The novel method is evaluated on bag-of-features based image classification problems.

• Periodical: Advanced Materials Research (Volumes 1006-1007)
• Pages: 764-767
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.1006-1007.764
• Online since: August 2014
• Authors: Hao Xiang Wang*, Shan Yue, Yang Li
• Keywords: Class Label, Image Classification, K-L Divergence, Vector Quantization

References

[1] T. Pham, M. Brandl, and D. Beck, Fuzzy declustering-based vector quantization, Pattern Recognition, vol. 42, no. 11, p.2570–2577, (2009).

DOI: 10.1016/j.patcog.2009.03.031

[2] K. Tasdemir, Vector quantization based approximate spectral clustering of large datasets, Pattern Recognition, vol. 45, no. 8, p.3034–3044, (2012).

DOI: 10.1016/j.patcog.2012.02.012

[3] H. J´egou, M. Douze, and C. Schmid, Improving bag-of-features for large scale image search, International Journal of Computer Vision, vol. 87, no. 3, p.316–336, (2010).

DOI: 10.1007/s11263-009-0285-2

[4] Y. -G. Jiang, C. -W. Ngo, and J. Yang, Towards optimal bag-of-features for object categorization and semantic video retrieval, 2007, p.494– 501, cited By (since 1996)99.

DOI: 10.1145/1282280.1282352

[5] J. Lai and Y. -C. Liaw, A novel encoding algorithm for vector quantization using transformed codebook, Pattern Recognition, vol. 42, no. 11, p.3065–3070, (2009).

DOI: 10.1016/j.patcog.2009.02.001

[6] Z. Lian, A. Godil, X. Sun, and J. Xiao, Cm-bof: visual similarity-based 3d shape retrieval using clock matching and bag-of-features, Machine Vision and Applications, p.1–20, (2013).

DOI: 10.1007/s00138-013-0501-5

[7] P. Sprechmann and G. Sapiro, Dictionary learning and sparse coding for unsupervised clustering, 2010, p.2042–(2045).

DOI: 10.21236/ada513140

[8] G. Yu, W. Russell, R. Schwartz, and J. Makhoul, Discriminant analysis and supervised vector quantization for continuous speech recognition, vol. 2, 1990, p.681–688.

DOI: 10.1109/icassp.1990.115850

[9] Z. Rached, F. Alajaji, and L. Campbell, The kullback-leibler divergence rate between markov sources, IEEE Transactions on Information Theory, vol. 50, no. 5, p.917–921, (2004).

DOI: 10.1109/tit.2004.826687

[10] S. Park and M. Shin, Kullback-leibler information of a censored variable and its applications, Statistics, (2013).

DOI: 10.1080/02331888.2013.800070

[11] J. J. -Y. Wang, H. Bensmail, and X. Gao, Joint learning and weighting of visual vocabulary for bag-of-feature based tissue classification, Pattern Recognition, vol. 46, no. 12, p.3249–3255, (2013).

DOI: 10.1016/j.patcog.2013.05.001

[12] D. Taniar and W. Rahayu, A taxonomy for nearest neighbour queries in spatial databases, Journal of Computer and System Sciences, vol. 79, no. 7, p.1017–1039, (2013).

DOI: 10.1016/j.jcss.2013.01.017

[13] S. Lazebnik, C. Schmid, and J. Ponce, Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories, vol. 2, 2006, p.2169–2178.

DOI: 10.1109/cvpr.2006.68

[14] J. Xu and H. Liu, Web user clustering analysis based on kmeans algorithm, vol. 2, (2010).