The structural and magnetic property of Gd5Si1.99Ge2Zn0.01and Gd5Si2Ge2 alloys prepared by arc-melting the starting materials with commercial available purity (99.95wt%) was investigated by x-ray diffraction and Vibrating Sample Magnetometer. The result shows that the Gd5Si1.99Ge2Zn0.01 alloy has monoclinic phase with Gd5Si2Ge2-type structure. The polymorphic orthorhombic phase with Gd5Si4-type structure coexists with the monoclinic phase in Gd5Si2Ge2 alloy. The addition of small Zn element in Gd5Si2Ge2 alloy results in a considerable enhancement of its magnetocaloric effect. The maximum magnetic entropy change rapidly increases from 5.03J/(kg K) to 20.70J/(kg K) for a magnetic field change from 0 to 1.5T. The magnetic order temperature is 278K in Gd5Si1.99Ge2Zn0.01 alloy and 278.5K in Gd5Si2Ge2 alloy respectively. The magnetocaloric effect of Gd5Si2Ge2 with the small addition of Zn is significantly improved.