Emotion Detection System Based on Speech and Facial Signals

Abstract:

This paper introduces the present status of speech emotion detection. In order to improve the emotion recognition rate of single mode, the bimodal fusion method based on speech and facial expression is proposed. First, we establishes emotional database include speech and facial expression. For different emotions, calm, happy, surprise, anger, sad, we extract ten speech parameters and use the PCA method to detect the speech emotion. Then we analyze the bimodal emotion detection of fusing facial expression information. The experiment results show that the emotion recognition rate with bimodal fusion is about 6 percent points higher than the recognition rate with only speech prosodic features

• Periodical: Advanced Materials Research (Volume 459)
• Pages: 483-487
• DOI: https://doi.org/10.4028/www.scientific.net/AMR.459.483
• Online since: January 2012
• Authors: Hui Zhao, Yu Tai Wang, Xing Hai Yang
• Keywords: Emotion Detection System, Facial Expression, Fusion Algorithm, Speech Feature

References

[1] Fu, L. Q., Wang, Y. B., Wang, C.Y., Voice emotion recognition based on multi-feature vector, Computer Science, vol. 36, No. 6, pp.231-234 (2009).

[2] Liu, X. W., Tan, H. C., Zhang, Y. J., New Research Advances in Facial Expression Recognition, Journal of Image and Graphics, vol. 11, No. 10, pp.1359-1368 (2006).

[3] Huang, Y. M., Zhang, G. B., Liu, H. B., Emotion Detection System Based on New Double-Mode Fusion Algorithm. Journal of Tianjin University, vol. 43, No. 12, pp.1067-1072 (2010).

[4] Huang, C. W., Jin, Z., Wang, Q. Y., Multi-modal emotion recognition based on speech signal and ecg signal, Journal of Southeast University, vol. 40, No. 5, pp.895-900 (2010).

[5] Yu, L. L., Cai, Z. X., Chen, M. Y., Speech Signal Emotional Characteristics Analysis and Recognition Research Overview, Journal of Circuits and Systems, vol. 12, No. 4, pp.77-84 (2007).

[6] Zhao, L. S., Zhang, Q., Wei, X. P., Survey on Speech Emotion Recognition, Journal of Application Research of Computers, vol. 26, No. 2, pp.428-432 (2009).

[7] Gong, T., Hu, T. S., Tian, X. Z., Human face expression recognition based on within-class modular PCA, Journal of Mechanical & Electrical Engineering Magazine, vol. 26, No. 7 pp.74-76 (2009).

[8] Xue, Y. L., Mao, X., Guo, Y., Research Advance of Facial Expression Recognition Human Computer Interaction, Journal of Image and Graphics, vol. 14, No. 5, pp.764-772 (2009).

[9] Huang, L., X., Xin, L., Zhao, L. Y., Bimodal Emotion Recognition Based on Adaptive Weights, Journal of Tsinghua University, Natural Science, vol. 48, No. S1, pp.715-719 (2008).

[10] Zhang, L. H., Yang, Y. C.: Emotional speech characteristics, Journal of Tsinghua University, vol. 48, No. S1, p.754–759 (2008).