Abstract: The defect of concrete structures on material level can be expressed by defect depth and defect range. Infrared thermal imaging technology of concrete material defect detection is actually an inverse problem of heat transfer. On the basis of the current research achievements, infrared thermal imaging methods used in concrete structure defects was deduced to a multi-objective function optimization problem. Considering the traditional optimization algorithm slow convergence speed and local minima faults, this paper introduce particle swarm optimization algorithm (PSO) and the BP neural network to detect concrete material defect depth and range.PSO algorithm was used to optimize neural networks connection weights between layers and the network topology. The simulation test results are in good agreement with the experiment results and verify the validity of this method.