It is difficult to process touching or broken characters in practical applications on optical character recognition. For touching or broken characters, a method based on mathematical morphology of binary image is put forward in the paper. On the basis of the relative theories of digital image processing, the overall process is introduced including separation of touching characters and connection of broken characters. First of all, character image is pre-processed through smoothing and threshold segmentation in order to generate binary image of characters. Then character regions which are touching or broken are processed through different operators of mathematical morphology of binary image by different structuring elements. Thus the touching characters are separated and broken characters are connected. For higher recognition rate, further processes are done to achieve normal and individual character regions.