Facial expression analysis is important in Human-Computer Interaction. How to improve the accuracy and objectivity of analysis should be concerned about. In this paper, a novel approach to context-aware based facial expression nets model is proposed. According to interactive environment’s characteristics and the relationship between context factors and facial expression features, CFEN model is constructed and applied to compute and analyze participant’s realistic facial expression in the context environment. It is possible to achieve facial expression’s knowledge representation and reasoning analysis. Experiment is designed and conducted to verify the theoretical model and approach proposed in this paper.