With the development of modern engineering such as micro-electro-mechanical system (MEMS) and new material, the deformation measurement of 3D displacement field is required. Moiré interferometry has become common for measuring the deformations of two-dimension in-plane displacement field. To solve the problem of 3D displacement field, a system for measuring the deformations of 3D displacement field is designed. By using this system, we can measure the out-of-plane displacement by appending a set of electronic speckle interference system based on Moiré interferometry system. Adjusting the incident angle of the incident light, the transmitting direction of the first order diffracted wave in a Moiré interferometry system is assigned to deviate with a small angle from the vertical direction of the specimen surface. Thus the first order diffracted wave is separated from the place of the appended interference field. The striated patterns of the deformations of 3D displacement field can be obtained at the same time. These striated patterns are recorded respectively by three CCD cameras and are stored and processed by a computer. The synchronous measuring method brings certain amount of measuring errors into the measurement of in-plane displacement. This measuring error is analyzed in theory. When real-time measurement is not required, this system can be changed into another system in which the in-plane and the out-of-plane displacement field are measured by time sharing to reduce the measuring errors. The theoretical derivation and a part of experimental results for verification are presented.