Vanishing point detection is a basic work in camera self-calibration, single view reconstruction and series of images matching. Our research is based on line segments clustering method. First, we scan the image with edge detection algorithm for series of line segments. Then, we construct a similar concept space to classify the segments according to the vector distances. At last, we can use each cluster of the line segments to estimate the responsible vanishing point. For the clusters of the line segments indicate the main direction in multiple lines, the detected vanishing points are principal direction points. From the experiments, we approve our algorithm can acquire accurate position of vanishing points in short time.