Kendall's tau is frequently used to meta-evaluate how well machine translation (MT) evaluation metrics score individual translations. Its focus on pairwise ...
May 23, 2023 · Kendall's tau is frequently used to meta-evaluate how well machine translation (MT) evaluation metrics score individual translations.
Instead of using Kendall's τ 𝜏 \tau italic_τ as the ranking-based meta-evaluation statistic, we propose to use a version of pairwise accuracy that includes ties ...
the methods discussed in the paper "Ties Matter: Modifying Kendall's Tau for Modern Metric Meta-Evaluation",. available at https://arxiv.org/abs/2305.14324.
Dec 10, 2023 · Kendall's τ is frequently used to meta-evaluate how well machine translation (MT) evaluation metrics score individual translations.
Missing: Tau | Show results with:Tau
Aug 14, 2023 · Ties Matter: Modifying Kendall's Tau for · Modern Metric Meta-Evaluation. Patrick Fernandes, Aman Madaan, Emmy Liu,. António Farinhas, Pedro ...
Ties Matter: Modifying Kendall's Tau for Modern Metric Meta-Evaluation Daniel Deutsch, George Foster, Markus Freitag EMNLP 2023 [pdf]; INSTRUCTSCORE: Towards ...
To evaluate automated metrics on GenAI-Bench, we follow TIFA160 [26] to report the Pearson and Kendall coefficients, which reflect the correlation of the ...
Missing: Tau | Show results with:Tau
Ties Matter: Meta-Evaluating Modern Metrics with Pairwise Accuracy and Tie Calibration ... We demonstrate that, in settings like modern MT meta-evaluation ...
Missing: Modifying | Show results with:Modifying
Ties Matter: Modifying Kendall's Tau for Modern Metric Meta-Evaluation. Kendall's tau is frequently used to meta-evaluate how well machine trans... 0 Daniel ...