Vibration sources, which caused the vibration of a shear-wall residential building, are inversed in this paper. In light of the test data, frequency-domain analysis is presented, and a finite element model is established. Furthermore, the time-domain acceleration responses of the structure are obtained. In compare with the British standard Vibration Dose Value (VDV) method, an Annoyance-Based Checking Procedure (ABCP) is proposed to evaluate the vibration of the building. The analysis indicates that both results agree reasonably well. A suggestion for structure engineers is proposed to consider the effect of environment vibration in the vicinity of the building.