For the disable people or the aged people, they need extra help in their daily household life. Various modern devices have been developed to improve their living quality. These devices always have automatic functions which are considered crucial for their independent life. Different to the traditional automatic household appliance such as manually infrared remote-controller and automatic voice-operated switch, this paper presents a novel approach based on computer vision to realize intelligent household control. This proposed system uses Adaboost and ASM algorithm for face detection and feature point calibration. Based on the facial key points, the head postures and mouth states are calculated and then coded into a special data packet. Finally, the packet is sent by an infrared carrier to the intelligent appliance with infrared receiver to control their operation manner. By using computer vision technology with infrared communication, the system is especially effective for the disable people such as the upper-body paraplegic and the people with hand dysfunction.