Aero-engine rotor system is the core component of engine. Aim at difficulties of fault diagnosis of engine rotor system, a method to detect the fault feature is proposed, which is based on recurrence plot (RP) and recurrence quantification analysis(RQA) by research of the characteristics and the mechanism of faults. An experiment is used to detect the fault of rotor system by using this new method. The results showed that the RQA is an effective way to extract features from vibration signal and by the use of quantitative features it is possible to identify and classify different types of rotor. Comparing with classical statistical features, the proposed algorithm has better classification rate. The research will be helpful in the further study of fault diagnosis of rotor system.