In natural medium, engineering materials and structures, it can be found that there are cavities everywhere. When structure is impacted by dynamic load, the scattering field will be produced because of the cavities, and it could cause dynamic stress concentration at the edge of the cavities. In this paper, the solution of displacement field for elastic semi-space with multiple cylindrical cavities by anti-plane SH-wave is constructed. In complex plane, considering the symmetry of SH-wave scattering, the displacement field aroused by the anti-plane SH-wave and the scattering displacement field impacted by the cylindrical cavities comprised of Fourier-Bessel series with undetermined coefficients which satisfies the stress-free condition on the ground surface are constructed. Through applying the method of multi-polar coordinate system, the equations with unknown coefficients can be obtained by using the stress free condition of the cylindrical cavities in the radial direction. According to orthogonality condition for trigonometric function, these equations can be reduced to a series of algebraic equations. Then the value of the unknown coefficients can be obtained by solving these algebraic equations. The total wave displacement field is the superposition of the displacement field aroused by the anti-plane SH-wave and the scattering displacement field. By using the expressions, an example is provided to show the effect of the change of relative location of the cylindrical cavities. Based on this solution, the problem of interaction of multiple cylindrical cavities and a linear crack in semi-space can be investigated further.