The CNC milling machine is extensively used in manufacturing of the die and the box-type parts. However, the tool errors, which caused by the cutting heat and the cutting force, seriously affect the machining accuracy of the machined parts. Furthermore, the tool errors are too complex to be calculated by the empirical formula. To solve this problem, a tool error compensation method is proposed in this paper. The least-square method is employed to structure the error model. A weighting coefficient is proposed to adapt the various working conditions. The macro program is used to realize the real-time error compensation. Using the tool error compensation method, the maximum tool error is reduced from 0.053 mm to 0.005 mm. The error is compensated by 90.5% compared with no compensation.