In this paper, an image processing and recognition system of the coal and gangue has been studied. We adopt digital video processing technology of high performance---DaVinci technology to design the system, which is made up by the chip DM6437 and some peripheral auxiliary equipment. The system takes advantage of the high performance of the chip to process the real-time image collected by the CCD camera. The system uses optimized image processing and recognition algorithms, and makes the recognition of coal and gangue by the different characteristic of their histograms. In the system, the histogram and processing image of coal and gangue can be displayed on the monitor.