Article

Free access

Immediate-head parsing for language models

Author:

Eugene CharniakAuthors Info & Claims

ACL '01: Proceedings of the 39th Annual Meeting on Association for Computational Linguistics

Pages 124 - 131

https://doi.org/10.3115/1073012.1073029

Published: 06 July 2001 Publication History

Abstract

We present two language models based upon an "immediate-head" parser --- our name for a parser that conditions all events below a constituent c upon the head of c. While all of the most accurate statistical parsers are of the immediate-head variety, no previous grammatical language model uses this technology. The perplexity for both of these models significantly improve upon the trigram model base-line as well as the best previous grammar-based language model. For the better of our two models these improvements are 24% and 14% respectively. We also suggest that improvement of the underlying parser should significantly improve the model's perplexity and that even in the near term there is a lot of potential for improvement in immediate-head language models.

References

[1]

Bod, R. What is the minimal set of fragments that achieves maximal parse accuracy. In Proceedings of Association for Computational Linguistics 2001. 2001.

Digital Library

[2]

Charniak, E. Treebank grammars. In Proceedings of the Thirteenth National Conference on Artificial Intelligence. AAAI Press/MIT Press, Menlo Park, 1996, 1031--1036.

Digital Library

[3]

Charniak, E. A maximum-entropy-inspired parser. In Proceedings of the 2000 Conference of the North American Chapter of the Association for Computational Linguistics. ACL, New Brunswick NJ, 2000.

Digital Library

[4]

Chelba, C. and Jelinek, F. Exploiting syntactic structure for language modeling. In Proceedings for COLING-ACL 98. ACL, New Brunswick NJ, 1998, 225--231.

Digital Library

[5]

Chi, Z. and Geman, S. Estimation of probabilistic context-free grammars. Computational Linguistics 242 (1998), 299--306.

Digital Library

[6]

Collins, M. J. Three generative lexicalized models for statistical parsing. In Proceedings of the 35th Annual Meeting of the ACL. 1997, 16--23.

Digital Library

[7]

Collins, M. J. Head-Driven Statistical Models for Natural Language Parsing. University of Pennsylvania, Ph.D. Dissertation, 1999.

Digital Library

[8]

Collins, M. J. Discriminative reranking for natural language parsing. In Proceedings of the International Conference on Machine Learning (ICML 2000). 2000.

Digital Library

[9]

Goddeau, D. Using probabilistic shift-reduce parsing in speech recognition systems. In Proceedings of the 2nd International Conference on Spoken Language Processing. 1992, 321--324.

[10]

Goodman, J. Putting it all together: language model combination. In ICASSP-2000. 2000.

[11]

Lauer, M. Corpus statistics meet the noun compound: some empirical results. In Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics. 1995, 47--55.

Digital Library

[12]

Magerman, D. M. Statistical decision-tree models for parsing. In Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics. 1995, 276--283.

Digital Library

[13]

Marcus, M. P., Santorini, B. and Marcinkiewicz, M. A. Building a large annotated corpus of English: the Penn treebank. Computational Linguistics 19 (1993), 313--330.

Digital Library

[14]

Ratnaparkhi, A. Learning to parse natural language with maximum entropy models. Machine Learning 34 1/2/3 (1999), 151--176.

Digital Library

[15]

Roark, B. Probabilistic top-down parsing and language modeling. Computational Linguistics (forthcoming).

Digital Library

[16]

Stolcke, A. An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics 21 (1995), 165--202.

Digital Library

[17]

Stolcke, A. and Segal, J. Precise ngram probabilities from stochastic context-free grammars. In Proceedings of the 32th Annual Meeting of the Association for Computational Linguistics. 1994, 74--79.

Digital Library

Cited By

Gobbel GReeves RJayaramaraja SGiuse DSperoff TBrown SElkin PMatheny M(2014)Development and evaluation of RapTATJournal of Biomedical Informatics10.1016/j.jbi.2013.11.00848:C(54-65)Online publication date: 1-Apr-2014
https://dl.acm.org/doi/10.1016/j.jbi.2013.11.008
Corazza ALavelli ASatta G(2013)An information-theoretic measure to evaluate parsing difficulty across treebanksACM Transactions on Speech and Language Processing 10.1145/2407736.24077379:4(1-31)Online publication date: 30-Jan-2013
https://dl.acm.org/doi/10.1145/2407736.2407737
Pauls AKlein DLi HLin COsborne M(2012)Large-scale syntactic language modeling with treeletsProceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 110.5555/2390524.2390654(959-968)Online publication date: 8-Jul-2012
https://dl.acm.org/doi/10.5555/2390524.2390654
Show More Cited By

Immediate-head parsing for language models
1. Computing methodologies
  1. Artificial intelligence
2. Hardware
  1. Power and energy
    1. Power estimation and optimization

Recommendations

Head-Driven Statistical Models for Natural Language Parsing

This article describes three statistical models for natural language parsing. The models extend methods from probabilistic context-free grammars to lexicalized grammars, leading to approaches in which a parse tree is represented as the sequence of ...
LLLR parsing
SAC '13: Proceedings of the 28th Annual ACM Symposium on Applied Computing

The idea of an LLLR parsing is presented. An LLLR(k) parser can be constructed for any LR(k) grammar but it produces the left parse of the input string in linear time (in respect to the length of the derivation) without backtracking. If used as a basis ...
GLR parsing with multiple grammars for natural language queries

This article presents an approach for parsing natural language queries that integrates multiple subparsers and subgrammars, in contrast to the traditional single grammar and parser approach. In using LR(k) parsers for natural language processing, we are ...

Comments

Information & Contributors

Information

Published In

cover image DL Hosted proceedings

ACL '01: Proceedings of the 39th Annual Meeting on Association for Computational Linguistics

July 2001

562 pages

General Chair:
Bonnie Lynn Webber

Publisher

Association for Computational Linguistics

United States

Publication History

Published: 06 July 2001

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 85 of 443 submissions, 19%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

89
Total Citations
View Citations
708
Total Downloads

Downloads (Last 12 months)84
Downloads (Last 6 weeks)5

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gobbel GReeves RJayaramaraja SGiuse DSperoff TBrown SElkin PMatheny M(2014)Development and evaluation of RapTATJournal of Biomedical Informatics10.1016/j.jbi.2013.11.00848:C(54-65)Online publication date: 1-Apr-2014
https://dl.acm.org/doi/10.1016/j.jbi.2013.11.008
Corazza ALavelli ASatta G(2013)An information-theoretic measure to evaluate parsing difficulty across treebanksACM Transactions on Speech and Language Processing 10.1145/2407736.24077379:4(1-31)Online publication date: 30-Jan-2013
https://dl.acm.org/doi/10.1145/2407736.2407737
Pauls AKlein DLi HLin COsborne M(2012)Large-scale syntactic language modeling with treeletsProceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers - Volume 110.5555/2390524.2390654(959-968)Online publication date: 8-Jul-2012
https://dl.acm.org/doi/10.5555/2390524.2390654
Liu DZhao ZHu YQian L(2012)Incorporating lexical semantic similarity to tree kernel-based chinese relation extractionProceedings of the 13th Chinese conference on Chinese Lexical Semantics10.1007/978-3-642-36337-5_2(11-21)Online publication date: 6-Jul-2012
https://dl.acm.org/doi/10.1007/978-3-642-36337-5_2
Beck Dde Medeiros Caseli H(2012)Bayesian induction of syntactic language models for brazilian portugueseProceedings of the 10th international conference on Computational Processing of the Portuguese Language10.1007/978-3-642-28885-2_18(157-167)Online publication date: 17-Apr-2012
https://dl.acm.org/doi/10.1007/978-3-642-28885-2_18
Meyers AKosaka MLiao SXue NWU DAPIDIANAKI MCARPUAT MSPECIA L(2011)Improving MT word alignment using aligned multi-stage parsesProceedings of the Fifth Workshop on Syntax, Semantics and Structure in Statistical Translation10.5555/2024261.2024271(88-97)Online publication date: 23-Jun-2011
https://dl.acm.org/doi/10.5555/2024261.2024271
Tan MZhou WZheng LWang SLin D(2011)A large scale distributed syntactic, semantic and lexical language model for machine translationProceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 110.5555/2002472.2002499(201-210)Online publication date: 19-Jun-2011
https://dl.acm.org/doi/10.5555/2002472.2002499
Zhou GZhu Q(2011)Kernel-based semantic relation detection and classification via enriched parse tree structureJournal of Computer Science and Technology10.5555/1991836.199184226:1(45-56)Online publication date: 1-Jan-2011
https://dl.acm.org/doi/10.5555/1991836.1991842
Zhou GKong F(2011)Learning noun phrase anaphoricity in coreference resolution via label propagationJournal of Computer Science and Technology10.5555/1991836.199184126:1(34-44)Online publication date: 1-Jan-2011
https://dl.acm.org/doi/10.5555/1991836.1991841
Xiao TZhu JZhu M(2011)Language Modeling for Syntax-Based Machine Translation Using Tree Substitution GrammarsACM Transactions on Asian Language Information Processing10.1145/2025384.202538610:4(1-29)Online publication date: 1-Dec-2011
https://dl.acm.org/doi/10.1145/2025384.2025386
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten