Purpose: To discuss a setup verification method based on computer stereo vision for the repeat setup in breast cancer fractional irradiation. Methods and Materials: A photogrammetric system is composed of two cameras and one computer. First, multiple characteristic markers’ coordinates are gotten by the two cameras, then the three-dimensional coordinates of the markers are calculated according to the basic principle of binocular calculated, which can construct breast and chest surface stereo features shape, thus the setup error can judge and correct before radiation therapy. At the tracking process, we use SITF (Scale Invariant Feature Transform) as the registration algorithm, which has strong robustness and accurate matching performance, meanwhile, dynamic choice matching image and local searching strategy are used in the process of calculation and mergence in order to make the target image matching more precisely. Results: Experimental results show that in breast cancer fractional irradiation, the system can accurately display setup error and can achieve real-time calculation. Conclusion: The proposed method can reduce setup error in breast cancer fractional irradiation, and has good stability and high precision.