We describe the design and construction about the Ultrasonic method to determine the defects in concrete structure. The wave velocity and its energy are different when it transfers in the concrete material with various defects. The defects can then be detected according to the relative wave velocity and energy variations in concrete material. In the present paper, two methods contain both the wavelet analysis and the nerve network used to determine the relative wave energy change in concrete, and then we will evaluate the defects in material by energy change. The result shows that the present method can give an accurate measurement for the defects in concrete structure.