research-article

Feature Extraction Method for Lip-reading under Variant Lighting Conditions

Authors:

Xinjun Ma,

Hongjun Zhang,

Yuanyuan LiAuthors Info & Claims

ICMLC '17: Proceedings of the 9th International Conference on Machine Learning and Computing

Pages 320 - 326

https://doi.org/10.1145/3055635.3056576

Published: 24 February 2017 Publication History

Get Access

Abstract

Automatic lip-reading is a technique of understanding the uttered speech by visually interpreting the lip movement of the speaker. with the development of the lip-reading, more and more related technologies are proposed. However, the current research of lip-reading is mainly conducted under the ideal lighting conditions and there is few researchers focus on the lip-reading technique under variant lighting conditions. For this problem, this paper proposes a new method of lip feature extraction under variant lighting conditions. The method consists of a preprocessing chain of illumination normalization and improved LBP features, which can improve the recognition rate of lip-reading under variant lighting conditions from two aspects. Experiments show that the lip feature extraction method proposed by this paper is effective.

References

[1]

Zhao, G., Barnard, M., & Pietikainen, M. (2009). Lipreading with local spatiotemporal descriptors. IEEE Transactions on Multimedia, 11(7), 1254--1265.

Digital Library

Google Scholar

[2]

Aleksic, P. S., & Katsaggelos, A. K. (2006). Audio-visual biometrics. Proceedings of the IEEE, 94(11), 2025--2044.

Crossref

Google Scholar

[3]

Almajai, I., Cox, S., Harvey, R., & Lan, Y. (2016). Improved Speaker Independent Lipreading using Speaker Adaptive Training and Deep Neural Networks. ICASSP.

Google Scholar

[4]

Bear, H. L., Harvey, R. W., & Lan, Y. (2015). Finding phonemes: improving machine lip-reading. Joint Conference on Facial Analysis, Animation and Auditory-Visual Speech Processing.

Google Scholar

[5]

ALan, Y., Harvey, R., Theobald, B. J., Bowden, R., & Ong, E. J. (2010). Improving visual features for lip-reading. International Conference on Auditory-Visual Speech Processing (pp. 142--147).

Google Scholar

[6]

Bakry, A., & Elgammal, A. (2013). Mkpls: manifold kernel partial least squares for lipreading and speaker identification. 684--691.

Digital Library

Google Scholar

[7]

Zhao, G., & Pietikainen, M. (2013). Visual Speaker Identification with Spatiotemporal Directional Features. Image Analysis and Recognition. Springer Berlin Heidelberg.

Google Scholar

[8]

M.Z. Ibrahim, & D.J. Mulvaney. (2015). Geometrical-based lip-reading using template probabilistic multi-dimension dynamic time warping &star;.Journal of Visual Communication & Image Representation, 30(C), 219--233.

Digital Library

Google Scholar

[9]

Lucey, P., & Sridharan, S. (2008). A visual front-end for a continuous pose-invariant lipreading system. International Conference on Signal Processing and Communication Systems (pp. 1--6). IEEE.

Crossref

Google Scholar

[10]

Shin, J., Jin, L., & Kim, D. (2011). Real-time lip reading system for isolated korean word recognition. Pattern Recognition, 44(3), 559--571.

Digital Library

Google Scholar

[11]

Ma X, Zhang H. Lip segmentation algorithm based on bi-color space{C}// Control Conference. IEEE, 2015.

Google Scholar

[12]

Morade, S. S., & Patnaik, S. (2015). Comparison of classifiers for lip reading with cuave and tulips database. Optik - International Journal for Light and Electron Optics, 126(24), 5753--5761.

Crossref

Google Scholar

[13]

Shi, Xiaoxing. "A New Time Wrapping Algorithm and Its Application on Neural Network Based Speech Recognition." Journal of Southeast Univwrsity (1999).

Google Scholar

Cited By

View all

Kulkarni AKirange D(2024)Towards Automated Lip Reading: Developing Marathi Lip Reading Datasets and Neural Network Frameworks2023 4th International Conference on Intelligent Technologies (CONIT)10.1109/CONIT61985.2024.10627583(1-6)Online publication date: 21-Jun-2024
https://doi.org/10.1109/CONIT61985.2024.10627583
Santos CCunha ACoelho P(2023)A Review on Deep Learning-Based Automatic LipreadingWireless Mobile Communication and Healthcare10.1007/978-3-031-32029-3_17(180-195)Online publication date: 14-May-2023
https://doi.org/10.1007/978-3-031-32029-3_17

Recommendations

Lip Reading using Simple Dynamic Features and a Novel ROI for Feature Extraction
SPML '18: Proceedings of the 2018 International Conference on Signal Processing and Machine Learning

Deaf or hard-of-hearing people mostly rely on lip-reading to understand speech. They demonstrate the ability of humans to understand speech from visual cues only. Automatic lip reading systems work in a similar fashion - by obtaining speech or text from ...
Visual units and confusion modelling for automatic lip-reading

Automatic lip-reading (ALR) is a challenging task because the visual speech signal is known to be missing some important information, such as voicing. We propose an approach to ALR that acknowledges that this information is missing but assumes that it ...
Analysis of the visual Lombard effect and automatic recognition experiments

This study focuses on automatic visual speech recognition in the presence of noise. The authors show that, when speech is produced in noisy environments, articulatory changes occur because of the Lombard effect; these changes are both audible and ...

Comments

Information & Contributors

Information

Published In

ICMLC '17: Proceedings of the 9th International Conference on Machine Learning and Computing

February 2017

545 pages

ISBN:9781450348171

DOI:10.1145/3055635

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Southwest Jiaotong University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 February 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICMLC 2017

ICMLC 2017: 2017 the 9th International Conference on Machine Learning and Computing

February 24 - 26, 2017

Singapore, Singapore

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
140
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Kulkarni AKirange D(2024)Towards Automated Lip Reading: Developing Marathi Lip Reading Datasets and Neural Network Frameworks2023 4th International Conference on Intelligent Technologies (CONIT)10.1109/CONIT61985.2024.10627583(1-6)Online publication date: 21-Jun-2024
https://doi.org/10.1109/CONIT61985.2024.10627583
Santos CCunha ACoelho P(2023)A Review on Deep Learning-Based Automatic LipreadingWireless Mobile Communication and Healthcare10.1007/978-3-031-32029-3_17(180-195)Online publication date: 14-May-2023
https://doi.org/10.1007/978-3-031-32029-3_17

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Recommendations

Lip Reading using Simple Dynamic Features and a Novel ROI for Feature Extraction

Visual units and confusion modelling for automatic lip-reading

Analysis of the visual Lombard effect and automatic recognition experiments

Comments

Information

Published In

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations