Classification is one of most important tasks in pattern recognition. Support vector machine (SVM) and its improved algorithms are quite important to classification. Although these approaches perform well in practice, they can’t take within-class and between-class scatter into consideration. Inspired from the Fisher’s discriminant ratio in linear discriminant analysis (LDA), a minimum within-class and maximum between-class scatter support vector machine (MMSVM) is proposed. MMSVM has the advantages of SVM and LDA and reflects distributions of classes, therefore, classification accuracy is improved in most cases. Experiments on many datasets verify the effectiveness of the proposed algorithm.