In this paper, we propose a machine vision based approach for detecting and classifying irregular low-contrast surface defects of segment magnet. The constituent material of it is ferrite which varies from silver gray to black in color .For this reason, the defects embedded in a low-contrast surface show no big different from its surrounding region, and even worse, all the surfaces and chamfers of segment magnet must be inspected. Our system is able to analyze all surfaces under inspection, to discover and classify its defects by means of image processing algorithms and support vector machine (SVM). A working prototype of the system has been built and tested to validate the proposed approach and to reproduce the difficult issues of the inspection system. The developed prototype includes three subsystems: an array of several CCD area cameras (Fig.1); a controllable roller LED light source(Fig.1); and a PC-based image processing system. The detection of the defects is performed by means of Canny edge detection, morphology and other feature extraction operations. The image processing and classification results demonstrate that the proposed method can identify surface defects effectively.