A stereo visual servo robotic system is developed on Nios II SOPC developing board with ALTERA FPGA chip to manipulate a retrofitted Mitsubishi RV-M2 robotic system. The 3-D position information between the target and stereo vision system can be extracted by low cost CMOS stereo vision algorithm first. Then, the relative motion between the robotic end-effector and the target can be planned to guide robot arm to catch the object. The fuzzy sliding mode control algorithm is employed to monitor the trajectory motion of each joint. The experimental results show that this visual servo robotic system can track and catch a moving target in 3D space and execute some interaction functions with player.