In this paper, we address the problem of automatic inspection complex surface using an ultrasonic technique and the reliability of inspection. First, an automatic ultrasonic inspection manipulator is introduced. Through ultrasonic inspection and defect analysis, the parameters of flaws can be calculated automatically. Then, a POD analysis method is provided to assess the reliability of inspection. Finally, Experiments are conducted and the POD curves with the sizes of flaws and the scanning gaps are obtained.