This paper presents a real microstructure model which has the same fiber volume fraction and tows’ spatial configuration with 3D rectangular composites to simulate the ballistic impact damage of the composites struck by steel projectile. The commercial available FEM code of Ls-Dyna was employed to calculate the interaction between the composite targets and steel projectile. From the comparison of residual velocities between simulation and experiment, it is proven the microstructure model can simulate the ballistic penetration with higher precision than the continuum model. The acceleration vs. time curve reveals the complicated interaction between composite and projectile in ballistic penetration. The prominent advantage of the microstructure model is that it can simulate the local damage mode of the composites at real microstructure level and obtain vivid simulating results.