Neural Classification of Linguistic Coherence using Long Short-Term Memories
Pages 28 - 31
Abstract
Given a set of sentences, a sentence orderer permutes the sentences in a way that the final text is linguistically coherent and semantically understandable. In this work, we focus on the binary and ternary tasks of ordering a pair of sentences regarding their linguistic coherence. We propose a methodology to automatically collect and annotate sentence ordering corpora in the news domain for English and German documents. Furthermore, we introduce a data-driven end-to-end neural architecture to learn the order of a pair of sentences and also recognize the cases where no ordering can be determined due to missing context.
References
[1]
X. Chen, X. Qiu, and X. Huang. Neural Sentence Ordering. ArXiv e-prints, July 2016.
[2]
F. Chollet. Keras. https://github.com/fchollet/keras, 2015.
[3]
V. Fromkin, R. Rodman, and N. Hyams. An Introduction to Language. Cengage Learning, 2010.
[4]
P. Golik, P. Doetsch, and H. Ney. Cross-Entropy vs. Squared Error Training: a Theoretical and Experimental Comparison. In Interspeech, pages 1756--1760, Aug. 2013.
[5]
G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580, 2012.
[6]
S. Hochreiter and J. Schmidhuber. Long Short-Term Memory. Neural Comput., 9(8):1735--1780, Nov. 1997.
[7]
E. H. Hovy. Planning Coherent Multisentential Text. In Proceedings of the 26th Annual Meeting on Association for Computational Linguistics, ACL '88, pages 163--169. Association for Computational Linguistics, 1988.
[8]
J. Li and D. Jurafsky. Neural Net Models for Open-Domain Discourse Coherence. CoRR, abs/1606.01545, 2016.
[9]
Z. Lin, H. T. Ng, and M.-Y. Kan. Automatically Evaluating Text Coherence Using Discourse Relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1, HLT '11, pages 997--1006. Association for Computational Linguistics, 2011.
[10]
W. Liu, X. Luo, J. Xuan, Z. Xu, and D. Jiang. Cognitive Memory-inspired Sentence Ordering Model. Knowledge-Based Systems, 104:1--13, 2016.
[11]
T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient Estimation of Word Representations in Vector Space. CoRR, abs/1301.3781, 2013.
[12]
P. Modaresi and S. Conrad. On Definition of Automatic Text Summarization. In Proceedings of Second International Conference on Digital Information Processing, Data Mining, and Wireless Communications, DIPDMWC2015, pages 33--40. SDIWC, 2015.
[13]
P. Modaresi and S. Conrad. Simurg: An Extendable Multilingual Corpus for Abstractive Single Document Summarization. In Proceedings of the 8th Forum for Information Retrieval Evaluation, FIRE '16. ACM, 2016.
[14]
R. Pascanu, T. Mikolov, and Y. Bengio. On the Difficulty of Training Recurrent Neural Networks. CoRR, abs/1211.5063, 2012.
[15]
J. Pennington, R. Socher, and C. D. Manning. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532--1543, 2014.
[16]
S. Verberne, L. Boves, N. Oostdijk, and P.-A. Coppen. Evaluating Discourse-based Answer Extraction for Why-question Answering. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '07, pages 735--736. ACM, 2007.
[17]
W. Zaremba, I. Sutskever, and O. Vinyals. Recurrent Neural Network Regularization. CoRR, abs/1409.2329, 2014.
[18]
M. D. Zeiler. ADADELTA: An Adaptive Learning Rate Method. CoRR, abs/1212.5701, 2012.
[19]
R. Zhang. Sentence Ordering Driven by Local and Global Coherence for Summary Generation. In Proceedings of the ACL 2011 Student Session, HLT-SS '11, pages 6--11. Association for Computational Linguistics, 2011.
Recommendations
Stop words detection using a long short term memory recurrent neural network
ICIT '21: Proceedings of the 2021 9th International Conference on Information Technology: IoT and Smart CityNatural language processing is a field of computer science that focuses on understanding and analyzing textual data in any given language. Analyzing textual data is very tedious and leads to erroneous results due to unnecessary and noisy data in the ...
Attention-based Stacked Bidirectional Long Short-term Memory Model for Word Sense Disambiguation
Word sense disambiguation is a basic task in Natural Language Processing which aims to identify the most appropriate sense of ambiguous words in different contexts by applying algorithm models. In this work, we propose a model that uses a stacked ...
Deep sentence embedding using long short-term memory networks: analysis and application to information retrieval
This paper develops a model that addresses sentence embedding, a hot topic in current natural language processing research, using recurrent neural networks (RNN) with Long Short-Term Memory (LSTM) cells. The proposed LSTM-RNN model sequentially takes ...
Comments
Information & Contributors
Information
Published In
December 2016
47 pages
ISBN:9781450348386
DOI:10.1145/3015157
Copyright © 2016 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
In-Cooperation
- Indian Statistical Institute, Kolkata: Indian Statistical Institute, Kolkata
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 08 December 2016
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
FIRE '16
Acceptance Rates
FIRE '16 Paper Acceptance Rate 7 of 22 submissions, 32%;
Overall Acceptance Rate 19 of 64 submissions, 30%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 114Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in