The Training Algorithm of Fuzzy Coupled Hidden Markov Models

Abstract:

A variety of coupled hidden Markov models (CHMMs) have recently been proposed as extensions of HMM to better characterize multiple interdependent sequences. The resulting models have multiple state variables that are temporally coupled via matrices of conditional probabilities. A generalised fuzzy approach to statistical modelling techniques is proposed in this paper. Fuzzy C-means (FCM) and fuzzy entropy (FE) techniques are combined into a generalised fuzzy technique and applied to coupled hidden Markov models. The CHMM based on the fuzzy c-means (FCM) and fuzzy entropy (FE) is referred to as FCM-FE-CHMM in this paper. By building up a generalised fuzzy objective function, several new formulae solving Training algorithms are theoretically derived for FCM-FE-CHMM. The fuzzy modelling techniques are very flexible since the degree of fuzziness, the degree of fuzzy entropy.

• Periodical: Applied Mechanics and Materials (Volumes 568-570)
• Pages: 254-259
• DOI: https://doi.org/10.4028/www.scientific.net/AMM.568-570.254
• Online since: June 2014
• Authors: Shi Ping Du*, Jian Wang, Yu Ming Wei
• Keywords: Baum-Welch Algorithm, Coupled hidden Markov models, Forward-Backward Algorithm, Fuzzy C-Means, Fuzzy Entropy, Hidden Markov Model (HMM)

References

[1] R. Cole,L. Hirschman, L. Atlas,M. Beckman, et al., The challenge of spoken language systems: Research directions for the nineties, IEEE Trans. Speech Audio Process. 1995, 3(1): 1-21.

DOI: 10.1109/89.365385

[2] L.R. Rabiner, A tutorial on hidden Markov models and selected applications in speech recognition, Proc IEEE. 1989, 77(2): 257-286.

DOI: 10.1109/5.18626

[3] L.R. Rabiner, B.H. Juang, Fundamentals of speech recognition, Prentice-Hall, (1993).

[4] Matthew Brand, Nuria Oliver, and Alex Pentland. Coupled hidden Markov models for complex action recognition. In Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition. 1997, pp.994-999.

DOI: 10.1109/cvpr.1997.609450

[5] S.R. Eddy. Profile hidden Markov models. Bioinformatics. 1998, 14(9): 755–763.

DOI: 10.1093/bioinformatics/14.9.755

[6] R. Durbin,S. Eddy,A. Krogh, G. Mitchison, Biological sequence analysis. Cambridge University Press, (1998).

[7] K. Asai, S. Hayamizu ,K. Handa, Prediction of protein secondary structure by the hidden Markov model, Comput. Appl. Biosci. 1993, 9(2): 141-146.

DOI: 10.1093/bioinformatics/9.2.141

[8] J. Kwon and K. Murphy. Modeling freeway traffic with coupled HMMs. Technical report, University of California at Berkeley, May (2000).

[9] I. Rezek and S. J. Roberts. Estimation of coupled hidden Markov models with application to biosignal interaction modelling. In Proc. IEEE Int. Conf. on Neural Network for Signal Processing. 2000, volume 2, p.804–813.

DOI: 10.1109/nnsp.2000.890160

[10] I. Rezek, M. Gibbs and S. J. Roberts. Maximum a posteriori estimation of coupled hidden Markov models. Journal of VLSI Signal Processing. 2002, 32(1-2): 55-66.

[11] Matthew Brand. Coupled hidden Markov models for modeling interactive processes. Technical Report 405, MIT Media Lab, (1997).

[12] A. V. Nefian, L. H. Liang, X. X. Liu, X. Pi, C. Mao, and K. Murphy. A coupled HMM for audio-visual speech recognition, in Proc. IEEE ICASSP, 2002, vol. 2, p.2013–(2016).

DOI: 10.1109/icassp.2002.1006167

[13] T.T. Kristjansson B.J. Frey, and T. Huang. Event-coupled hidden Markov models. In Proc. IEEE Int. Conf. on Multimedia and Exposition. 2000, vol. 1, p.385–388.

[14] Z. Ghahramani and M. I. Jordan. Factorial hidden Markov models. Machine Learning. 1997, 29(2-3): 245–275.

[15] Y. Bengio and P. Frasconi. Input-Output HMMs for sequence processing. IEEE Trans. Neural Networks. 1996, 7(5): 1231–1249.

DOI: 10.1109/72.536317

[16] R.N. Dave, R. Krishnapuram , Robust clustering methods: a unified view , IEEE Trans. Fuzzy Syst, Vol. 5, No., p.270–293, (1997).

DOI: 10.1109/91.580801

[17] Dat Tran and Michael Wagner, Fuzzy hidden Markov models for speech and speaker recognition, in Proc. NAFIPS'99, pp.426-430, USA, (1999).

DOI: 10.1109/nafips.1999.781728

[18] J.C. Bezdek, Pattern recognition with fuzzy objective function algorithms, Plenum Press, New York and London, (1981).

[19] Dat Tran and Michael Wagner, Fuzzy entropy clustering, the FUZZ-IEEE'2000 Conf., vol. 1, pp.152-158, USA, (2000).