A new sliding mode controller for trajectory tracking of ball and plate system is proposed. In the controller, a nonlinear observer which is independent on plant model is used to observer the uncertainties and disturbance of the system. This paper proved that the nonlinear observer is convergence by properly selecting the parameters of observer. Sliding mode controller based on nonlinear observer is designed for the ball and plate system and it can guarantee stabilization of closed-loop system. The results of simulation and experiments indicate that the proposed nonlinear observer can observer uncertainties and disturbance of system, the proposed sliding mode control method is effectively to solve the problems of trajectory tracking in the nonlinear ,uncertain system.