Training Data Reduction and Classification Based on Greedy Kernel Principal Component Analysis and Fuzzy C-Means Algorithm

Abstract:

Nonlinear feature extraction used standard Kernel Principal Component Analysis (KPCA) method has large memories and high computational complexity in large datasets. A Greedy Kernel Principal Component Analysis (GKPCA) method is applied to reduce training data and deal with the nonlinear feature extraction problem for training data of large data in classification. First, a subset, which approximates to the original training data, is selected from the full training data using the greedy technique of the GKPCA method. Then, the feature extraction model is trained by the subset instead of the full training data. Finally, FCM algorithm classifies feature extraction data of the GKPCA, KPCA and PCA methods, respectively. The simulation results indicate that the feature extraction performance of both the GKPCA, and KPCA methods outperform the PCA method. In addition of retaining the performance of the KPCA method, the GKPCA method reduces computational complexity due to the reduced training set in classification.

• Periodical: Applied Mechanics and Materials (Volumes 347-350)
• Pages: 2390-2394
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.347-350.2390
• Online since: August 2013
• Authors: Xiao Fang Liu, Chun Yang
• Keywords: Classification, Fuzzy C-Means Algorithm, Greedy Kernel Principal Component Analysis, Kernel Matrix, Nonlinear Feature Extraction, Training Data Reduction

References

[1] B. Schölkopf, A. Smola, and K. R. Muller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Computation, vol. 10, p.1299–1319, July (1998).

DOI: 10.1162/089976698300017467

[2] B. Schölkopf and A. J. Smola, Learning with Kernels, Cambridge: MIT Press, 2002, p.35–71.

[3] V. Franc and V. Hlaváč, Greedy algorithm for a training set reduction in the kernel methods, In N. Petkov and M. A. Westenberg, editors, Computer Analysis of Images and Patterns, Berlin: Springer-Verlag, 2003, p.426–433.

DOI: 10.1007/978-3-540-45179-2_53

[4] V. Franc, Optimization algorithms for kernel methods, PhD thesis, Prague: Czech Technical University, 2005, p.87–103.

[5] T. Tangkuampien and D. Suter, Human Motion De-noising via Greedy Kernel Principal Component Analysis Filtering, 18th International Conference on Pattern Recognition (ICPR 06) , Pattern Recognition, 2006, p.457–460.

DOI: 10.1109/icpr.2006.639

[6] J. C. Bezdek and W. F. Ehrlich, FCM: The fuzzy c-mean clustering algorithm, Computers and Geoscienc., vol. 10, February 1984, p.191–203.

DOI: 10.1016/0098-3004(84)90020-7

[7] A. Asuncion and D. J. Newman. UCI Machine learning repository. (2012-12-10). http: /www. ics. uci. edu/~mlearn/MLRepository. html. Irvine, CA: University of California, School of Information and Computer Science, (2007).

[8] N. R. Pal and J. C. Bezdek, On cluster validity for the fuzzy c-means model, IEEE Trans, Fuzzy Systems, vol. 3, August 1995, p.370–373.

DOI: 10.1109/91.413225

[9] J. C. Bezdek, Cluster validity with fuzzy sets, Journal Cybernet, vol. 3, February 1974, p.58–73.

[10] X. Xie and G. Beni, A validity measures for fuzzy clustering, IEEE Transactions on Pattern Analysis and Machine Intelligence. vol. 13, August 1991, p.841–747.

DOI: 10.1109/34.85677