Saadia Gabriel, Asli Celikyilmaz, Rahul Jha, Yejin Choi, and Jianfeng Gao. 2021. GO FIGURE: A Meta Evaluation of Factuality in Summarization. In Findings of the ...
Oct 24, 2020 · In this paper, we introduce GO FIGURE, a meta-evaluation framework for evaluating factuality evaluation metrics. We propose five necessary and ...
Aug 1, 2021 · We propose five necessary conditions to evaluate factuality metrics on di- agnostic factuality data across three different summarization tasks.
GO FIGURE, a meta-evaluation framework for evaluating factuality evaluation metrics, is introduced and it is revealed that while QA metrics generally ...
Experiments show that our models significantly boost the factual consistency of system-generated summaries without sacrificing summary quality in terms of both ...
GO FIGURE: A Meta Evaluation of Factuality in Summarization. Interactive data viewing of evaluation datasets. To view auto-generated data: python load_data.py ...
In this paper, we propose a new method called ClozE to evaluate factual consistency by cloze model, instantiated based on masked language model(MLM), ...
Oct 24, 2020 · We introduce five necessary, common-sense conditions for effective factuality metrics and experiment with nine recent factuality metrics using ...
People also ask
What is the evaluation of meta?
What is meta analysis in evaluation?
GO FIGURE: A meta evaluation of factuality in summarization. S Gabriel, A Celikyilmaz, R Jha, Y Choi, J Gao. arXiv preprint arXiv:2010.12834, 2020. 83, 2020.
Nov 15, 2022 · (2021) proposed a meta-evaluation framework, GO Figure, which evaluates the sen- sitivity and validity of factual consistency metrics with only ...