multiple object tracking is an active and important research topic. It faces many challenging problems. Object extraction and data association are two most key steps in multiple object tracking. To improve tracking performance, this paper proposed a tracking method which combines Kalman filter and energy minimization-based data association. Moving objects are segmented through frame difference. Its can be consider as the vertex. All detections in adjacent frames are be used to construct a graph. The energy is finally minimized with a graph cuts optimization. Data association can be consider as multiple labeling problems. Object corresponding can be obtained through energy minimization. Experiment results demonstrate this method can be accurately tracking two moving objects.