The study presents a human tracking system. To tracking a person, we adopt a particle filter as tracking kernel, since the method has proven successful for tracking in non-linear and non-Gaussian estimation. In a particle filter, a set of weighted particles represents the possible target sates. In this study, we measure the weight according to both the appearances of the target object and background scene to improve the discriminability between them. In our tracker, the appearances are modeled as color histogram, since it is scale and rotation invariant. However, the color histogram extraction for a large number of overlap regions is repeated redundantly and inefficiently. To speed up it, we reduce the cost for calculating overlapped regions by creating a cumulative histogram map for the processing image. The experimental results show that the tracker has the best precision improvement, and the tracking speed is 49.7 fps for 384 × 288 resolution, when we use 600 particles. The results show that the proposed method can be applied to a real-time human tracking system with high precision.