In this paper, visco-elastic-plasticity theory is employed to establish a nonlinear finite element model of the asphalt mixture pavement. The influence of pavement structure, the ability of rutting resistance of middle layer and traffic load on shear stress distribution for asphalt pavement are discussed. The numerical results show that shear stress analysis can be used to analyze the rutting of asphalt pavement. The asphalt materials of middle layer have a great impact on rutting and shear stress. Modified asphalt is a useful middle layer material to decrease the rutting, and the hard asphalt is also an economical material to reduce rutting. Overload and overpressure easily cause pavement rutting damage. Pavement longitudinal grade is not the main reason leading to rutting at the long slope, but brake frequently in the long slope is the real cause of rutting.