A portable three-dimensional vision coordinate measuring system is proposed in this paper. It consists of a light pen, a CCD camera and a laptop computer. The light pen is a hand-held probe with three point light sources using LEDs and a spherical touch stylus arranged in one line. The CCD camera takes the image of LEDs on the light pen and the 3D coordinates of measured point at the center of touch stylus are obtained by computing the positions of LEDs in the image when the probe contacts the object surface. The equations of the vision measurement system are derived by using perspective of three collinear points and spatial analytic geometry. The measuring device can be set up easily on an industrial site and the measurement is convenient. The object surface can be simple, complex, or hidden from the view of camera. The effectiveness of the mathematical model and validity of the measuring system are confirmed by experiments.