Article

Free access

Discriminative syntactic language modeling for speech recognition

Authors:

Michael Collins,

Murat SaraclarAuthors Info & Claims

ACL '05: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics

Pages 507 - 514

https://doi.org/10.3115/1219840.1219903

Published: 25 June 2005 Publication History

Abstract

We describe a method for discriminative training of a language model that makes use of syntactic features. We follow a reranking approach, where a baseline recogniser is used to produce 1000-best output for each acoustic input, and a second "reranking" model is then used to choose an utterance from these 1000-best lists. The reranking model makes use of syntactic features together with a parameter estimation method that is based on the perception algorithm. We describe experiments on the Switchboard speech recognition task. The syntactic features provide an additional 0.3% reduction in test-set error rate beyond the model of (Roark et al., 2004a; Roark et al., 2004b) (significant at p < 0.001), which makes use of a discriminatively trained n-gram model, giving a total reduction of 1.2% over the baseline Switchboard system.

References

[1]

Eugene Charniak. 2001. Immediate-head parsing for language models. In Proc. ACL.

Digital Library

[2]

Ciprian Chelba and Frederick Jelinek. 1998. Exploiting syntactic structure for language modeling. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, pages 225--231.

Digital Library

[3]

Ciprian Chelba and Frederick Jelinek. 2000. Structured language modeling. Computer Speech and Language, 14(4):283--332.

Digital Library

[4]

Ciprian Chelba. 2000. Exploiting Syntactic Structure for Natural Language Modeling. Ph.D. thesis, The Johns Hopkins University.

Digital Library

[5]

Stanley Chen and Joshua Goodman. 1998. An empirical study of smoothing techniques for language modeling. Technical Report, TR-10-98, Harvard University.

[6]

Michael J. Collins. 1999. Head-Driven Statistical Models for Natural Language Parsing. Ph.D. thesis, University of Pennsylvania.

Digital Library

[7]

Michael Collins. 2002. Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In Proc. EMNLP, pages 1--8.

Digital Library

[8]

Michael Collins. 2004. Parameter estimation for statistical parsing models: Theory and practice of distribution-free methods. In Harry Bunt, John Carroll, and Giorgio Satta, editors, New Developments in Parsing Technology. Kluwer Academic Publishers, Dordrecht.

Digital Library

[9]

Frederick Jelinek and John Lafferty. 1991. Computation of the probability of initial substring generation by stochastic context-free grammars. Computational Linguistics, 17(3):315--323.

Digital Library

[10]

Mark Johnson, Stuart Geman, Steven Canon, Zhiyi Chi, and Stefan Riezler. 1999. Estimators for stochastic "unifi cation-based" grammars. In Proc. ACL, pages 535--541.

Digital Library

[11]

Daniel Jurafsky, Chuck Wooters, Jonathan Segal, Andreas Stolcke, Eric Fosler, Gary Tajchman, and Nelson Morgan. 1995. Using a stochastic context-free grammar as a language model for speech recognition. In Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing, pages 189--192.

[12]

John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. ICML, pages 282--289, Williams College, Williamstown, MA, USA.

Digital Library

[13]

Andrej Ljolje, Enrico Bocchieri, Michael Riley, Brian Roark, Murat Saraclar, and Izhak Shafran. 2003. The AT&T 1xRT CTS system. In Rich Transcription Workshop.

[14]

Franz Josef Och, Daniel Gildea, Sanjeev Khudanpur, Anoop Sarkar, Kenji Yamada, Alex Fraser, Shankar Kumar, Libin Shen, David Smith, Katherine Eng, Viren Jain, Zhen Jin, and Dragomir Radev. 2004. A smorgasbord of features for statistical machine translation. In Proceedings of HLT-NAACL 2004.

[15]

Brian Roark, Murat Saraclar, and Michael Collins. 2004a. Corrective language modeling for large vocabulary ASR with the perceptron algorithm. In Proc. ICASSP, pages 749--752.

[16]

Brian Roark, Murat Saraclar, Michael Collins, and Mark Johnson. 2004b. Discriminative language modeling with conditional random fields and the perceptron algorithm. In Proc. ACL.

Digital Library

[17]

Brian Roark, Murat Saraclar, and Michael Collins. 2005. Discriminative n-gram language modeling. Computer Speech and Language, submitted.

Digital Library

[18]

Brian Roark. 2001a. Probabilistic top-down parsing and language modeling. Computational Linguistics, 27(2):249--276.

Digital Library

[19]

Brian Roark. 2001b. Robust Probabilistic Predictive Syntactic Processing. Ph.D. thesis, Brown University. http://arXiv.org/abs/cs/0105019.

Digital Library

[20]

Ronald Rosenfeld, Stanley Chen, and Xiaojin Zhu. 2001. Whole-sentence exponential language models: a vehicle for linguistic-statistical integration. In Computer Speech and Language.

[21]

Fei Sha and Fernando Pereira. 2003. Shallow parsing with conditional random fields. In Proceedings of the Human Language Technology Conference and Meeting of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL), Edmonton, Canada.

Digital Library

[22]

Andreas Stolcke and Jonathan Segal. 1994. Precise n-gram probabilities from stochastic context-free grammars. In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics, pages 74--79.

Digital Library

[23]

Andreas Stolcke. 1995. An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics, 21(2):165--202.

Digital Library

[24]

Wen Wang and Mary P. Harper. 2002. The superARV language model: Investigating the effectiveness of tightly integrating multiple knowledge sources. In Proc. EMNLP, pages 238--247.

Digital Library

[25]

Wen Wang, Andreas Stolcke, and Mary P. Harper. 2004. The use of a linguistically motivated language model in conversational speech recognition. In Proc. ICASSP.

[26]

Wen Wang. 2003. Statistical parsing and language modeling based on constraint dependency grammar. Ph.D. thesis, Purdue University.

Digital Library

[27]

Peng Xu, Ciprian Chelba, and Frederick Jelinek. 2002. A study on richer syntactic dependencies for structured language modeling. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 191--198.

Digital Library

[28]

Peng Xu, Ahmad Emami, and Frederick Jelinek. 2003. Training connectionist models for the structured language model. In Proc. EMNLP, pages 160--167.

Digital Library

Cited By

Rastrow AKhudanpur SDredze M(2012)Revisiting the case for explicit syntactic information in language modelsProceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT10.5555/2390940.2390947(50-58)Online publication date: 8-Jun-2012
https://dl.acm.org/doi/10.5555/2390940.2390947
Hai Son LAllauzen AYvon F(2012)Measuring the influence of long range dependencies with neural network language modelsProceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT10.5555/2390940.2390941(1-10)Online publication date: 8-Jun-2012
https://dl.acm.org/doi/10.5555/2390940.2390941
Rastrow ADredze MKhudanpur SLi HLin COsborne M(2012)Fast syntactic analysis for statistical language modeling via substructure sharing and uptrainingProceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 110.5555/2390524.2390550(175-183)Online publication date: 8-Jul-2012
https://dl.acm.org/doi/10.5555/2390524.2390550
Show More Cited By

Discriminative syntactic language modeling for speech recognition
1. Computing methodologies
  1. Artificial intelligence
2. Hardware
  1. Power and energy
    1. Power estimation and optimization

Recommendations

Cross-lingual language modeling with syntactic reordering for low-resource speech recognition
EMNLP-CoNLL '12: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning

This paper proposes cross-lingual language modeling for transcribing source resource-poor languages and translating them into target resource-rich languages if necessary. Our focus is to improve the speech recognition performance of low-resource ...
Syntactic discriminative language model rerankers for statistical machine translation

This article describes a method that successfully exploits syntactic features for n-best translation candidate reranking using perceptrons. We motivate the utility of syntax by demonstrating the superior performance of parsers over n-gram language ...
Combined speech enhancement and auditory modelling for robust distributed speech recognition

The performance of automatic speech recognition (ASR) systems in the presence of noise is an area that has attracted a lot of research interest. Additive noise from interfering noise sources, and convolutional noise arising from transmission channel ...

Comments

Information & Contributors

Information

Published In

cover image DL Hosted proceedings

ACL '05: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics

June 2005

657 pages

General Chair:
Kevin Knight
University of Southern California

Publisher

Association for Computational Linguistics

United States

Publication History

Published: 25 June 2005

Qualifiers

Article

Acceptance Rates

ACL '05 Paper Acceptance Rate 77 of 423 submissions, 18%;

Overall Acceptance Rate 85 of 443 submissions, 19%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
439
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)3

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Rastrow AKhudanpur SDredze M(2012)Revisiting the case for explicit syntactic information in language modelsProceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT10.5555/2390940.2390947(50-58)Online publication date: 8-Jun-2012
https://dl.acm.org/doi/10.5555/2390940.2390947
Hai Son LAllauzen AYvon F(2012)Measuring the influence of long range dependencies with neural network language modelsProceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT10.5555/2390940.2390941(1-10)Online publication date: 8-Jun-2012
https://dl.acm.org/doi/10.5555/2390940.2390941
Rastrow ADredze MKhudanpur SLi HLin COsborne M(2012)Fast syntactic analysis for statistical language modeling via substructure sharing and uptrainingProceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 110.5555/2390524.2390550(175-183)Online publication date: 8-Jul-2012
https://dl.acm.org/doi/10.5555/2390524.2390550
Beck Dde Medeiros Caseli H(2012)Bayesian induction of syntactic language models for brazilian portugueseProceedings of the 10th international conference on Computational Processing of the Portuguese Language10.1007/978-3-642-28885-2_18(157-167)Online publication date: 17-Apr-2012
https://dl.acm.org/doi/10.1007/978-3-642-28885-2_18
Monz CMerlo PBarzilay RJohnson M(2011)Statistical machine translation with local language modelsProceedings of the Conference on Empirical Methods in Natural Language Processing10.5555/2145432.2145528(869-879)Online publication date: 27-Jul-2011
https://dl.acm.org/doi/10.5555/2145432.2145528
Schwartz LCallison-Burch CSchuler WWu SLin D(2011)Incremental syntactic language models for phrase-based translationProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 110.5555/2002472.2002552(620-631)Online publication date: 19-Jun-2011
https://dl.acm.org/doi/10.5555/2002472.2002552
Wang WMcKeown KJoshi AHuang CJurafsky D(2010)"Got you!"Proceedings of the 23rd International Conference on Computational Linguistics10.5555/1873781.1873910(1146-1154)Online publication date: 23-Aug-2010
https://dl.acm.org/doi/10.5555/1873781.1873910
Jha MAndreas JThadani KRosenthal SMcKeown KCallison-Burch CDredze M(2010)Corpus creation for new genresProceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk10.5555/1866696.1866698(13-20)Online publication date: 6-Jun-2010
https://dl.acm.org/doi/10.5555/1866696.1866698
Ghodke SBird SKaplan R(2010)Fast query for large treebanksHuman Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics10.5555/1857999.1858033(267-275)Online publication date: 2-Jun-2010
https://dl.acm.org/doi/10.5555/1857999.1858033
Cherry CSuzuki HKoehn PMihalcea R(2009)Discriminative substring decoding for transliterationProceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 3 - Volume 310.5555/1699648.1699652(1066-1075)Online publication date: 6-Aug-2009
https://dl.acm.org/doi/10.5555/1699648.1699652
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten