Low tensile strength and high brittleness are the inherent defects of cement-based composites. So improving the toughness effectively is an urgent need for developing high-performance cement materials. The aim of this study is to study the toughening effect by mixing CaCO3 whiskers and polypropylene fibers in cement mortar. The results showed that CaCO3 whiskers could increase the split tensile strength of cement mortar of 28d age by 26.2% at maximum with the whisker content of 10wt.%; when CaCO3 whiskers content was 10wt.% and polypropylene fibers content was 0.20vol.%, the split tensile strength of cement mortar was increased by 21.1% at 28d age; when the fiber length was 19mm, the composite toughening effect of whiskers and fibers was more effective. The toughening mechanisms of CaCO3 whiskers and polypropylene fibers in cement mortar were also discussed.