Image classification poses challenges to retrieval technology. Though the Support Vector Machine (SVM) has been successfully applied to pattern recognition, its performance is limited by the feature space and parameters in the training process. Our work thus has two central themes. Construct the optimum feature space for training SVM from image features extraction by nonlinear dimensionality reduction based on manifold learning, and meanwhile establish the RBF kernel based SVM classifier by training with the best parameters with a global search capacity of the Quantum-behaved Particle Swarm Optimization (QPSO). Experiments show that our model not only improves the learning ability, but also significantly enhances the accuracy of image classification.