In rotary cutting difficult-to-cut materials, less tool wear is observed at higher cutting speeds, contrary to the case of conventional single-point turning. In order to examine this characteristic feature of rotary cutting, this study focuses on the effects of cutting speed and tool peripheral speed on tool failure. Dry rotary cutting tests of stainless steel SUS304 were carried out using coated carbide tools. Cutting speeds were set at 100 m/min, the speed typically recommended for conventional turning, and 500 m/min. At 100 m/min, adhesions and subsequent chippings were observed on the cutting edges. Meanwhile, at 500 m/min the higher average temperature at the tool-workpiece contact area caused small wear spots, rather than adhesions at the optimized tool speed. Hence, for rotary cutting, a higher cutting speed is more effective in terms of both achieving several-times-higher productivity and reducing the amount of tool failure due to the difference of tool failure modes.