Unmanned surface vehicle (USV) system has been one of main research directions in mobile robotics because it can be used in many situations. However, high performance path following control, especially straight line tracking control, has been one of the difficult problems in autonomous control of USV system. In this paper, we propose a new straight line path following control algorithm by combining yaw angle feedback and back-stepping technique and show its closed loop stability. The most absorbing advantage of the proposed controller is that it not only reserve the good performance of back-stepping controller but also bring much faster convergent rate, which is very important in real applications. The simulation results with respect to a training ship model have shown the feasibility and validity of the proposed method.