Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systems

Italo Luis Da Silva, Hanqi Yan, Lin Gui, Yulan He


Abstract
The inherent ambiguity of cause and effect boundaries poses a challenge in evaluating causal event extraction tasks. Traditional metrics like Exact Match and BertScore poorly reflect model performance, so we trained evaluation models to approximate human evaluation, achieving high agreement. We used them to perform Reinforcement Learning with extraction models to align them with human preference, prioritising semantic understanding. We successfully explored our approach through multiple datasets, including transferring an evaluator trained on one dataset to another as a way to decrease the reliance on human-annotated data. In that vein, we also propose a weak-to-strong supervision method that uses a fraction of the annotated data to train an evaluation model while still achieving high performance in training an RL model.
Anthology ID:
2024.emnlp-main.816
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
14707–14719
Language:
URL:
https://aclanthology.org/2024.emnlp-main.816
DOI:
10.18653/v1/2024.emnlp-main.816
Bibkey:
Cite (ACL):
Italo Luis Da Silva, Hanqi Yan, Lin Gui, and Yulan He. 2024. Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systems. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 14707–14719, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Weak Reward Model Transforms Generative Models into Robust Causal Event Extraction Systems (Silva et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-main.816.pdf
Software:
 2024.emnlp-main.816.software.tgz
Data:
 2024.emnlp-main.816.data.tgz