Aimed at the problems such as welding stress concentration and especially irregular distribution of welding residual stress for anchorage area in cable-stayed bridges, three full-scale beam-anchor-plate connection models of Chongqing Jiangjin Guanyinyan Yangtse Rive bridge were made and the tests were performed. The purposes of the tests and study were to investigate the distribution of welding residual stress in anchorage area and moreover to find a way to reduce and eliminate the welding residual tensile stresses. For this reason ultrasonic impact was imposed on the main welds of the three models. The result of the experiment showed that there were quite considerable welding residual stresses in the anchorage area. It also displayed that the ultrasonic impact was able to effectively decrease and even release the welding residual tensile stress. The ultrasonic impact was able to not only improve the adverse state of the welds stress but also make the distribution of the welds stress smoother. The ultrasonic impact is an effective and feasible method to decrease and eliminate welding residual stress and thereby improve the fatigue strength of welds for the steel bridge.