Rotary ultrasonic machining (RUM) is considered to be a very effective and relatively accurate way to drill deep holes in brittle materials. Although brittle fracture (micro chipping) is the dominant material removal mechanism utilized by the RUM process, poor surface roughness and deep penetrated cracks are the consequence if the machining parameters are not properly controlled. To ensure the quality of the generated surface and to improve the process efficiency, efforts have been made in this study to correlate the material removal mechanisms, surface integrity and tool wear involved in the RUM process. Diamond-impregnated tools were used in the experiment and the ultrasonic vibration frequency was kept at 20 kHz. Three major material removal modes namely, impact mode, grinding mode and erosion mode were found to be the dominant removal processes at the tool tip, around the diamond wheel and around the steel sleeve respectively. It was also found that, during the grinding/erosion processes, the bonding material of the wheel was first eroded away and left big part of diamond grits well-exposed. Pull-out and/or fracture are normally the consequence of these exposed diamond grits due to the lack of support and protection.