A precise approach for the tooth profile modification (TPM) of helical gear is presented in the paper based on the static contact finite element analysis (FEA). The high-precision finite element model of helical meshing gear pairs is established. The type and amount of TPM are accurately determined by the static contact FEA results. The dynamic contact simulations of helical gears with and without tooth modification are investigated to estimate the vibration reduction effect of the TPM. Moreover the numerical simulations are compared with the experimental results. Both results show that the proposed precise TPM of helical gears is effective on vibration reduction around the working load, and the dynamic contact simulation is effective on estimating the vibration reduction influence of the TPM.