Study on Dual Mode Fusion Method of Video and Audio

Li Juan Shi; Ping Feng; Jian Zhao; Li Rong Wang; Na Che

doi:10.4028/www.scientific.net/AMM.734.412

Paper Titles

The Intelligent CAD Design for the Quenching Machine Tool Based on the NEI in the Full Lifecycle
p.393

TTS Tape Script Construction and Apply to SCADA System
p.398

A Self-Management System Combined with Constructivist Theory
p.403

A Generation System of Intelligent Power Dispatching Operation Sheet
p.408

Study on Dual Mode Fusion Method of Video and Audio
p.412

A Rule-Based Opto-Electronic Detection and Target Recognition System
p.416

Design and Implement of Operational Rule Base Based on Machine Learning and Association Rule Mining
p.422

The Application of the Fuzzy Cognitive Map among Intravenous Drug Users
p.428

Research on Dynamic Parking Space Allocation Model in Closed Parking Lot
p.435

HomeApplied Mechanics and MaterialsApplied Mechanics and Materials Vol. 734Study on Dual Mode Fusion Method of Video and...

Study on Dual Mode Fusion Method of Video and Audio

Abstract:

In order to solve the hearing-impaired students in class only rely on sign language, amount of classroom information received less, This paper studies video and audio dual mode fusion algorithm combined with lip reading、speech recognition technology and information fusion technology.First ,speech feature extraction, processing of speech signal, the speech synchronization output text. At the same time, extraction of video features, voice and video signal fusion, Make voice information into visual information that the hearing-impaired students can receive. Make the students receive text messages as receive visual information, improve speech recognition rate, so meet the need of the classroom teaching for hearing-impaired students.

You might also be interested in these eBooks

Electronics, Automation and Engineering of Power Systems

View Preview

Info:

Periodical:

Applied Mechanics and Materials (Volume 734)

Pages:

412-415

DOI:

https://doi.org/10.4028/www.scientific.net/AMM.734.412

Citation:

Cite this paper

Online since:

February 2015

Authors:

Li Juan Shi, Ping Feng, Jian Zhao*, Li Rong Wang, Na Che

Keywords:

Audio Information, Classroom Teaching of Hearing-Impaired Student, Dual Mode, Fusion, Video Information

Export:

RIS, BibTeX

Price:

Permissions CCC:

Request Permissions

Permissions PLS:

Request Permissions

Сopyright:

Citation:

* - Corresponding Author

References

[1] B. Mark. E.J. Holden, Robyn Owens: Lip tracking using pattern matching snakes : ACCV2002 Publications, Australia. 2002. pp.1-6.

Google Scholar

[2] Quoc Dinh Nguyen , Maurice Milgram, Thi-Hoang-Lan Nguyen: Multi Features Models for Robust Lip Tracking: 2008 10th Intl. Conf. on Control, Automation, Robotics and Vision, Hanoi, Vietnam, 2008. pp.1333-1337.

DOI: 10.1109/icarcv.2008.4795715

Google Scholar

[3] Brice Beaumesnil , Franck Luthon: Real Time Tracking for 3D Realistic Lip Animation, Proceedings of the 18th International Conference on Pattern Recognition, Norwich, NY (2006).

DOI: 10.1109/icpr.2006.956

Google Scholar

[4] Zhiyan Han: Research on Robust Feature Extracting and Visualization of Speech Signal, Doctor, Northeastern University, 2009. In Chinese.

Google Scholar

[5] Lirong Wang, Jianlei. Wang: Lip contour modeling based on PCA. Optics and Precision Engineering Vol. 20 (2012) , pp.2775-2779. In Chinese.

Google Scholar