research-article

Deep Reinforcement Learning with Copy-oriented Context Awareness and Weighted Rewards for Abstractive Summarization

Author:

Caidong TanAuthors Info & Claims

CACML '23: Proceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning

Pages 84 - 89

https://doi.org/10.1145/3590003.3590019

Published: 29 May 2023 Publication History

Abstract

This paper presents a deep context-aware model with a copy mechanism based on reinforcement learning for abstractive text summarization. Our model is optimized using weighted ROUGEs as global prediction-based rewards and the self-critical policy gradient training algorithm, which can reduce the inconsistency between training and testing by directly optimizing the evaluation metrics. To alleviate the lexical diversity and component diversity problems caused by global prediction rewards, we improve the richness of the multi-head self-attention mechanism to capture context through global deep context representation with copy mechanism. We conduct experiments and demonstrate that our model outperforms many existing benchmarks over the Gigaword, LCSTS, and CNN/DM datasets. The experimental results demonstrate that our model has a significant effect on improving the quality of summarization.

References

[1]

Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2016. An actor-critic algorithm for sequence prediction. arXiv preprint arXiv:1607.07086 (2016).

[2]

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems 28 (2015).

[3]

C Chang, C Huang, and Jane Yungjen Hsu. 2018. A hybrid word-character model for abstractive summarization. arXiv preprint arXiv:1802.09968 (2018).

[4]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.

Digital Library

[5]

Baotian Hu, Qingcai Chen, and Fangze Zhu. 2015. Lcsts: A large scale chinese short text summarization dataset. arXiv preprint arXiv:1506.05865 (2015).

[6]

Luyang Huang, Lingfei Wu, and Lu Wang. 2020. Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward. arXiv preprint arXiv:2005.01159 (2020).

[7]

Yaser Keneshloo, Tian Shi, Naren Ramakrishnan, and Chandan K Reddy. 2019. Deep reinforcement learning for sequence-to-sequence models. IEEE transactions on neural networks and learning systems 31, 7 (2019), 2469–2489.

[8]

Wojciech Kryściński, Romain Paulus, Caiming Xiong, and Richard Socher. 2018. Improving abstraction in text summarization. arXiv preprint arXiv:1808.07913 (2018).

[9]

Alex M Lamb, Anirudh Goyal ALIAS PARTH GOYAL, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. 2016. Professor forcing: A new algorithm for training recurrent networks. Advances in neural information processing systems 29 (2016).

[10]

Siyao Li, Deren Lei, Pengda Qin, and William Yang Wang. 2019. Deep reinforcement learning with distributional semantic rewards for abstractive summarization. arXiv preprint arXiv:1909.00141 (2019).

[11]

Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74–81.

[12]

Yang Liu and Mirella Lapata. 2019. Text summarization with pretrained encoders. arXiv preprint arXiv:1908.08345 (2019).

[13]

Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304 (2017).

[14]

Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang, Jiusheng Chen, Ruofei Zhang, 2021. ProphetNet-x: large-scale pre-training models for English, Chinese, multi-lingual, dialog, and code generation. arXiv preprint arXiv:2104.08006 (2021).

[15]

Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2015. Sequence level training with recurrent neural networks. arXiv preprint arXiv:1511.06732 (2015).

[16]

Alexander M Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685 (2015).

[17]

Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368 (2017).

[18]

Eva Sharma, Luyang Huang, Zhe Hu, and Lu Wang. 2019. An entity-driven framework for abstractive summarization. arXiv preprint arXiv:1909.02059 (2019).

[19]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).

[20]

Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. Advances in neural information processing systems 28 (2015).

[21]

Li Wang, Junlin Yao, Yunzhe Tao, Li Zhong, Wei Liu, and Qiang Du. 2018. A reinforced topic-aware convolutional sequence-to-sequence model for abstractive text summarization. arXiv preprint arXiv:1805.03616 (2018).

[22]

Liqiang Xiao, Lu Wang, Hao He, and Yaohui Jin. 2020. Copy or rewrite: Hybrid summarization with hierarchical reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 9306–9313.

[23]

Baosong Yang, Jian Li, Derek F Wong, Lidia S Chao, Xing Wang, and Zhaopeng Tu. 2019. Context-aware self-attention networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 387–394.

Digital Library

[24]

Qingyu Zhou, Nan Yang, Furu Wei, and Ming Zhou. 2017. Selective encoding for abstractive sentence summarization. arXiv preprint arXiv:1704.07073 (2017).

Cited By

Won SKim Y(2023)A Study on the Health Index Based on Degradation Patterns in Time Series Data Using ProphetNet ModelJournal of Society of Korea Industrial and Systems Engineering10.11627/jksie.2023.46.3.12346:3(123-138)Online publication date: 30-Sep-2023
https://doi.org/10.11627/jksie.2023.46.3.123

Index Terms

Deep Reinforcement Learning with Copy-oriented Context Awareness and Weighted Rewards for Abstractive Summarization
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing

Recommendations

Deep Learning-Based Abstractive Summarization for Brazilian Portuguese Texts
Intelligent Systems
Abstract
Automatic summarization captures the most relevant information and condenses it into an understandable text in natural language. Such a task can be classified as either extractive or abstractive summarization. Research on Brazilian Portuguese-...
Abstractive Summarization Improved by WordNet-Based Extractive Sentences
Natural Language Processing and Chinese Computing
Abstract
Recently, the seq2seq abstractive summarization models have achieved good results on the CNN/Daily Mail dataset. Still, how to improve abstractive methods with extractive methods is a good research direction, since extractive methods have their ...
Cross-domain aspect/sentiment-aware abstractive review summarization by combining topic modeling and deep reinforcement learning
Abstract
Review text has been widely studied in traditional tasks such as sentiment analysis and aspect extraction. However, to date, no work is toward the end-to-end abstractive review summarization that is essential for business organizations and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CACML '23: Proceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning

March 2023

598 pages

ISBN:9781450399449

DOI:10.1145/3590003

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 May 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CACML 2023

CACML 2023: 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning

March 17 - 19, 2023

Shanghai, China

Acceptance Rates

CACML '23 Paper Acceptance Rate 93 of 241 submissions, 39%;

Overall Acceptance Rate 93 of 241 submissions, 39%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
33
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Won SKim Y(2023)A Study on the Health Index Based on Degradation Patterns in Time Series Data Using ProphetNet ModelJournal of Society of Korea Industrial and Systems Engineering10.11627/jksie.2023.46.3.12346:3(123-138)Online publication date: 30-Sep-2023
https://doi.org/10.11627/jksie.2023.46.3.123

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents