It is well known that the ductility of magnesium and magnesium alloy is too poor to be processed. After normal hot extrusion, the ductility of magnesium alloy will be improved, but not noticeably. In this paper, further extrusion (Extrusion Ratio is 6.25) is adopted to commercial AZ31 magnesium alloy rods (Extrusion Ratio is 2) at different temperatures in order to refine the size of grain, then the rods with the crystallite size between 4-10μm is obtained, and the influence of crystallite with different grain size on the ductility of this alloy is investigated further. A lot of experiments and research indicate that: the elongation of the initial billet with the grain size of 30μ m can reach about 12%; after extrusion at different temperatures, the grain size can be refined to below 10μm, and the ductility goes up to 24%-30%. It also shows that, when the grain size of AZ31 magnesium alloy is below 10μm, the ductility is effected obviously. Only when the grain size is about a few microns, the ductility of AZ31 magnesium alloy will be improved noticeably.