In die and mold industry, there is a tendency to use milling in the finishing machining of dies and molds as an alternative to the traditional EDM process, consequently the surface integrity in milling is considered as one of the most important indices. In this study, surface roughness, micrograph of machined surface, surface microhardness, residual stress, and metallurgical texture of the surface layer were considered. The influence of geometrical characteristics of cutting tools and tool wear on surface integrity was studied. The results showed that hard milling of hardened die steel could yield a quite satisfied surface integrity by proper process; compressive residual stress was induced on machined surface, and the compressive stress induced in transversal direction was almost 3 times of that in longitudinal direction; tool wear had a significant negative influence on surface finish and caused the machined surface soften. These conclusions revealed the effects of tool conditions on surface integrity and would play a practical significance in the machining of hardened die steel.