skip to main content
10.1145/3022198.3026357acmconferencesArticle/Chapter ViewAbstractPublication PagescscwConference Proceedingsconference-collections
poster

Possible Confounds in Word-based Semantic Similarity Test Data

Published: 25 February 2017 Publication History

Abstract

Semantic similarity or semantic relatedness are features of natural language that contribute to the challenge machines face when analyzing text. Although semantic relatedness is still a complex challenge only few ground truth data set exist. We argue that the available corpora used to evaluate the performance of natural language tools do not capture all elements of the phenomenon. We present a set of simple interventions that illustrate 1) framing effects influence similarity perception, 2) the distribution of similarity across multiple users is important and 3) semantic relatedness is asymmetric.

References

[1]
Alexander Budanitsky and Graeme Hirst. 2001. Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures. In Proc. Workshop on WordNet and Other Lexical Resources.
[2]
Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2002. Placing Search in Context: The Concept Revisited. ACM Transactions on Information Systems 20, 1 (2002).
[3]
Evgeniy Gabrilovich and Shaul Markovitch. 2007. Computing Semantic Relatedness using Wikipedia-based Explicit Semantic Analysis. In Proc. IJCAI '07. 1606--1611.
[4]
Jay J. Jiang and David W. Conrath. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. arXiv cmp-lg/9709008 (1997).
[5]
Philip Resnik. 1995. Using information content to evaluate semantic similarity in a taxonomy. In Proc. IJCAI '95. 448--453.
[6]
Michael Strube and Simone Paolo Ponzetto. 2006. WikiRelate! Computing Semantic Relatedness Using Wikipedia. In Proc. AAAI '16. 1419--1424.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CSCW '17 Companion: Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing
February 2017
472 pages
ISBN:9781450346887
DOI:10.1145/3022198
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 February 2017

Check for updates

Author Tags

  1. human computation
  2. semantic similarity

Qualifiers

  • Poster

Conference

CSCW '17
Sponsor:
CSCW '17: Computer Supported Cooperative Work and Social Computing
February 25 - March 1, 2017
Oregon, Portland, USA

Acceptance Rates

CSCW '17 Companion Paper Acceptance Rate 183 of 530 submissions, 35%;
Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)1
Reflects downloads up to 01 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media