This paper presents a new method of fault diagnosis for electro-hydraulic servo system using support vector machine based on robust observer. Firstly, an electro-hydraulic system is introduced and a linear robust observer is designed for the system. Secondly, inner leakage and outer leakage faults of the system are experimentally simulated. Simulating results show that robust residual is sensitive to leakage faults but can not distinguish inner leakage and outer leakage in rod side. Thirdly, a nonlinear support vector machine was closely studied by regarding waveform and kurtosis index of displacement, residual, and pressure. Finally, educating and testing experiments show that the nonlinear support vector based on robust observer can diagnose all leakage faults of the system successfully.