SynDG: Syntax-aware Dialogue Generation
Pages 504 - 510
Abstract
Dialogue system is designed to converse with humans in a natural way. As an essential part of dialogue system, dialogue generation aims to generate proper response given historical context. Recently, sequence-to-sequence (seq2seq) based models have achieved great success but suffer from ungrammatical problems. In this paper, we propose a Syntax-aware Dialogue Generation (SynDG) model that incorporates syntactic information to generate grammatical responses with an encoder-decoder framework. Specifically, we first construct a syntax-graph with a dependency parser on the dialogue corpus. Then, we employ three graph embedding algorithms to learn syntactic word representations as the input of seq2seq framework. Furthermore, we devise training strategies to predict syntactic structure of the sentence for sufficient syntax understanding. Our empirical study on two multi-turn dialogue datasets demonstrates the effectiveness of SynDG in generating natural and grammatical responses.
References
[1]
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).
[2]
Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems 26 (2013).
[3]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
[4]
Alice E Fischer and Frances S Grodzinsky. 1993. The anatomy of programming languages. Prentice Hall.
[5]
Xiang Gao, Sungjin Lee, Yizhe Zhang, Chris Brockett, Michel Galley, Jianfeng Gao, and William B Dolan. 2019. Jointly Optimizing Diversity and Relevance in Neural Response Generation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 1229–1238.
[6]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
[7]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[8]
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and William B Dolan. 2016. A Diversity-Promoting Objective Function for Neural Conversation Models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 110–119.
[9]
Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 986–995.
[10]
Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Twenty-ninth AAAI conference on artificial intelligence.
[11]
Longxuan Ma, Mingda Li, Wei-Nan Zhang, Jiapeng Li, and Ting Liu. 2021. Unstructured Text Enhanced Open-Domain Dialogue System: A Systematic Survey. ACM Transactions on Information Systems (TOIS) 40, 1 (2021), 1–44.
[12]
Nikita Moghe, Siddhartha Arora, Suman Banerjee, and Mitesh M Khapra. 2018. Towards Exploiting Background Knowledge for Building Conversation Systems. In EMNLP.
[13]
Bin Ni, Xiaolei Lu, and Yiqi Tong. 2021. SynXLM-R: Syntax-Enhanced XLM-R in Translation Quality Estimation. In CCF International Conference on Natural Language Processing and Chinese Computing. Springer, 27–40.
[14]
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311–318.
[15]
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532–1543.
[16]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
[17]
Ashwin Ram, Rohit Prasad, Chandra Khatri, Anu Venkatesh, Raefer Gabriel, Qing Liu, Jeff Nunn, Behnam Hedayatnia, Ming Cheng, Ashish Nagar, 2018. Conversational ai: The science behind the alexa prize. arXiv preprint arXiv:1801.03604 (2018).
[18]
Pengjie Ren, Zhumin Chen, Christof Monz, Jun Ma, and Maarten de Rijke. 2020. Thinking globally, acting locally: Distantly supervised global-to-local knowledge selection for background based conversation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 8697–8704.
[19]
Iulian Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30.
[20]
Heung-Yeung Shum, Xiao-dong He, and Di Li. 2018. From Eliza to XiaoIce: challenges and opportunities with social chatbots. Frontiers of Information Technology & Electronic Engineering 19, 1 (2018), 10–26.
[21]
Felix Stahlberg and Shankar Kumar. 2021. Synthetic data generation for grammatical error correction with tagged corruption models. arXiv preprint arXiv:2105.13318 (2021).
[22]
Yujin Takahashi, Satoru Katsumata, and Mamoru Komachi. 2020. Grammatical error correction using pseudo learner corpus considering learner’s error tendency. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop. 27–32.
[23]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. Advances in Neural Information Processing Systems 30 (2017), 5998–6008.
[24]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In International Conference on Learning Representations.
[25]
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. Advances in neural information processing systems 28 (2015).
[26]
Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAI conference on artificial intelligence, Vol. 28.
[27]
Joseph Weizenbaum. 1966. ELIZA—a computer program for the study of natural language communication between man and machine. Commun. ACM 9, 1 (1966), 36–45.
[28]
Jiarui Yao, Haoling Qiu, Jin Zhao, Bonan Min, and Nianwen Xue. 2021. Factuality assessment as modal dependency parsing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 1540–1550.
[29]
Yuxiao Ye and Simone Teufel. 2021. End-to-end argument mining as biaffine dependency parsing. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 669–678.
[30]
Meishan Zhang. 2020. A survey of syntactic-semantic parsing based on constituent and dependency structures. Science China Technological Sciences 63, 10 (2020), 1898–1920.
[31]
Wen Zheng and Ke Zhou. 2019. Enhancing conversational dialogue models with grounded knowledge. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 709–718.
Index Terms
- SynDG: Syntax-aware Dialogue Generation
Recommendations
Grammatical Error Correction with Dependency Distance
CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge ManagementGrammatical Error Correction (GEC) task is always considered as low resource machine translation task which translates a sentence in an ungrammatical language to a grammatical language. As the state-of-the-art approach to GEC task, transformer-based ...
SOGO: A Social Intelligent Negotiation Dialogue System
IVA '18: Proceedings of the 18th International Conference on Intelligent Virtual AgentsIn this paper, we propose a semi-automatic social intelligent negotiation dialogue system that interweaves task utterance with conversational strategies to engage human users in negotiation. Our two-phase system operates sequentially in a reasoning-and-...
Syntax-Aware Sentence Matching with Graph Convolutional Networks
Knowledge Science, Engineering and ManagementAbstractNatural language sentence matching, as a fundamental technology for a variety of tasks, plays a key role in many natural language processing systems. In this article, we propose a new method which incorporates syntactic structure into “matching-...
Comments
Information & Contributors
Information
Published In
March 2023
824 pages
ISBN:9781450399029
DOI:10.1145/3594315
Copyright © 2023 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 02 August 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCAI 2023
ICCAI 2023: 2023 9th International Conference on Computing and Artificial Intelligence
March 17 - 20, 2023
Tianjin, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 182Total Downloads
- Downloads (Last 12 months)142
- Downloads (Last 6 weeks)20
Reflects downloads up to 06 Jan 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in