<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hirao-etal-2018-automatic">
    <titleInfo>
        <title>Automatic Pyramid Evaluation Exploiting EDU-based Extractive Reference Summaries</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Tsutomu</namePart>
        <namePart type="family">Hirao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hidetaka</namePart>
        <namePart type="family">Kamigaito</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Masaaki</namePart>
        <namePart type="family">Nagata</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-oct-nov</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Ellen</namePart>
            <namePart type="family">Riloff</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">David</namePart>
            <namePart type="family">Chiang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Julia</namePart>
            <namePart type="family">Hockenmaier</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jun’ichi</namePart>
            <namePart type="family">Tsujii</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Brussels, Belgium</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper tackles automation of the pyramid method, a reliable manual evaluation framework. To construct a pyramid, we transform human-made reference summaries into extractive reference summaries that consist of Elementary Discourse Units (EDUs) obtained from source documents and then weight every EDU by counting the number of extractive reference summaries that contain the EDU. A summary is scored by the correspondences between EDUs in the summary and those in the pyramid. Experiments on DUC and TAC data sets show that our methods strongly correlate with various manual evaluations.</abstract>
    <identifier type="citekey">hirao-etal-2018-automatic</identifier>
    <identifier type="doi">10.18653/v1/D18-1450</identifier>
    <location>
        <url>https://aclanthology.org/D18-1450</url>
    </location>
    <part>
        <date>2018-oct-nov</date>
        <extent unit="page">
            <start>4177</start>
            <end>4186</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Automatic Pyramid Evaluation Exploiting EDU-based Extractive Reference Summaries
%A Hirao, Tsutomu
%A Kamigaito, Hidetaka
%A Nagata, Masaaki
%Y Riloff, Ellen
%Y Chiang, David
%Y Hockenmaier, Julia
%Y Tsujii, Jun’ichi
%S Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
%D 2018
%8 oct nov
%I Association for Computational Linguistics
%C Brussels, Belgium
%F hirao-etal-2018-automatic
%X This paper tackles automation of the pyramid method, a reliable manual evaluation framework. To construct a pyramid, we transform human-made reference summaries into extractive reference summaries that consist of Elementary Discourse Units (EDUs) obtained from source documents and then weight every EDU by counting the number of extractive reference summaries that contain the EDU. A summary is scored by the correspondences between EDUs in the summary and those in the pyramid. Experiments on DUC and TAC data sets show that our methods strongly correlate with various manual evaluations.
%R 10.18653/v1/D18-1450
%U https://aclanthology.org/D18-1450
%U https://doi.org/10.18653/v1/D18-1450
%P 4177-4186

Markdown (Informal)

[Automatic Pyramid Evaluation Exploiting EDU-based Extractive Reference Summaries](https://aclanthology.org/D18-1450) (Hirao et al., EMNLP 2018)

• Automatic Pyramid Evaluation Exploiting EDU-based Extractive Reference Summaries (Hirao et al., EMNLP 2018)

ACL

• Tsutomu Hirao, Hidetaka Kamigaito, and Masaaki Nagata. 2018. Automatic Pyramid Evaluation Exploiting EDU-based Extractive Reference Summaries. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4177–4186, Brussels, Belgium. Association for Computational Linguistics.