To develop the small engine used for the wireless control helicopter, a very high power density source will be required. The rotary engine (RE) is advantageous in the points such as a volume of the engine, the exhaust gas emissions, and the vibrations. In this research, in order to develop the small RE, the design of the combustion chamber was reviewed. Additionally, the method of the surface treatment of the rotor housing was investigated by the friction wear test to enable the material substitution for aluminum matrix composites. The friction wear tests of composite materials, the Cr and Ni-plated material were conducted. As a result, the sample with the most excellent abrasion resistance was the composite material reinforced with Al2O3 fibers. The wear-loss of this composite material was 1/10 compared with the Ni-plated material. This composite material formed oil pockets between the fiber and the matrix, and a friction coefficient fell below 0.1. Then, the aluminum rotor housing reinforced partially by Al2O3 fiber was fabricated using a low-pressure infiltration method. The maximum power of the small RE (1 rotor, 30 cc) which built in this housing reached about 5 PS in the range from 13,000 to 15,000 rpm. The inside of the housing used composite materials was not damaged even after 30 hours. On the other hand, the flaking off part was observed on the inside part in the case with the housing used the Ni-plated material. Additionally, the output and the torque of the RE made of composite materials were higher than those of the Ni-plated material. This was considered because the friction coefficient of composite materials was lower than that of the Ni-plated material. Finally, the small unmanned helicopter equipped with RE succeeded in a stable flight.