A 3D finite element model is established to simulate the ultrasonic shot peening process by using a finite element software ABAQUS. The residual stress distribution of the AISI 304 stainless steel induced by ultrasonic shot peening (USP) is predicted by finite element analysis. Ultrasonic shot peening (USP) process can cause a compressive residual stress layer on the surface of the material. During the simulation, many factors, e.g., ultrasonic shot peening duration, initial residual stress, hourglass, etc., are taken into consideration for the purpose of optimizing the process. The simulation results show that ultrasonic shot peening can produce a compressive residual stress layer on the surface of the material even if there is initial residual tensile stress (250MPa) and the longer peening duration. The residual stress of simulation were compared with the experiment data which were obtained under the same ultrasonic shot peening parameters and have a good agreement with the measurement values by X-ray diffraction method. In conclusion, ultrasonic shot peening is an effective method for protecting weldments against stress corrosion cracking by introducing the compressive residual stress layer into the surface of stainless steel.