By the finite element analysis software ABAQUS and the function of coupling process between heat and stress, the welding residual stress of Cr5Mo and 20 steel joint was analyzed. In addition the heat treatment of dissimilar steel welded joint was simulated. The residual stress distributions of dissimilar steel welding and heat treatment after welding were obtained. The comparison of welding residual stress between the homogenous steel and dissimilar steel was carried out. The results indicate that the welding residual stress of the same steel is lower than that of dissimilar steel welded joint obviously. Because of the difference of thermal expansion coefficient for base metal and welding microstructure, the relatively higher residual stress is produced due to the bigger thermal expansion coefficient of base metal. The highest annular residual stress is in welding line root of internal wall, while the highest axial residual stress is in welding line surface of outer wall. The welding residual stress of dissimilar welded joint is reduced obviously after heat treatment. The research results provide the possibility for optimizing the welding procedure and improving the reliability of dissimilar steel welding joint.