poster

Diverse retrieval via greedy optimization of expected 1-call@k in a latent subtopic relevance model

Authors:

Sadegh Kharazmi,

Sarvnaz KarimiAuthors Info & Claims

CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management

Pages 1977 - 1980

https://doi.org/10.1145/2063576.2063869

Published: 24 October 2011 Publication History

Abstract

It has been previously observed that optimization of the 1-call@k relevance objective (i.e., a set-based objective that is 1 if at least one document is relevant, otherwise 0) empirically correlates with diverse retrieval. In this paper, we proceed one step further and show theoretically that greedily optimizing expected 1-call@k w.r.t. a latent subtopic model of binary relevance leads to a diverse retrieval algorithm sharing many features of existing diversification approaches. This new result is complementary to a variety of diverse retrieval algorithms derived from alternate rank-based relevance criteria such as average precision and reciprocal rank. As such, the derivation presented here for expected 1-call@k provides a novel theoretical perspective on the emergence of diversity via a latent subtopic model of relevance --- an idea underlying both ambiguous and faceted subtopic retrieval that have been used to motivate diverse retrieval.

References

[1]

R. Agrawal, S. Gollapudi, A. Halverson, and S. Ieong. Diversifying search results. In WSDM'09, 5--14, ACM, 2009.

Digital Library

[2]

D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent Dirichlet allocation. Journal of Machine Learning Research (JLMR), 3:993--1022, 2003.

Digital Library

[3]

C. Buckley and E. M. Voorhees. Evaluating evaluation measure stability. In SIGIR'00, 33--40, ACM, 2000.

Digital Library

[4]

J. Carbonell and J. Goldstein. The use of MMR, diversity-based reranking for reording documents and producing summaries. In SIGIR'98, 335--336, ACM, 1998.

Digital Library

[5]

O. Chapelle, D. Metzler, Y. Zhang, and P. Grinspan. Expected reciprocal rank for graded relevance. In CIKM'09, 621--630, ACM, 2009.

Digital Library

[6]

H. Chen and D. R. Karger. Less is more: Probabilistic models for retrieving fewer relevant documents. In SIGIR'06, 429--436, ACM, 2006.

Digital Library

[7]

C. Clarke, M. Kolla, G. Cormack, O. Vechtomova, A. Ashkan, S. Büttcher, and I. MacKinnon. Novelty and diversity in information retrieval evaluation. In SIGIR'08, 659--666, ACM, 2008.

Digital Library

[8]

S. Deerwester, S. T. Dumaisand, G. W. Furnas, T. K. Landauer, and R. Harshman. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6):391--407, 1990.

[9]

W. Goffman. On relevance as a measure. Information Storage and Retrieval, 2(3):201--203, 1964.

[10]

S. Guo and S. Sanner. Probabilistic latent maximal marginal relevance. In SIGIR'10, 833--834, ACM, 2010.

Digital Library

[11]

T. Jebara, R. Kondor, and A. Howard. Probability product kernels. Journal of Machine Learning Research (JLMR), 5:819--844, 2004.

Digital Library

[12]

S. Robertson and S. Walker. Some simple approximations to the 2-poisson model for probabilistic weighted retrieval. In SIGIR'94, 345--354, ACM, 1994.

Digital Library

[13]

G. Salton and M. McGill. Introduction to modern information retrieval. McGraw-Hill, 1983.

Digital Library

[14]

R. L. Santos, C. Macdonald, and I. Ounis. Exploiting query reformulations for web search result diversification. In WWW'10, 881--890, ACM, 2010.

Digital Library

[15]

E. M. Voorhees. TREC-8 question answering track report. In Proceedings of the 8th Text Retrieval Conference, pages 77--82, 1999.

[16]

J. Wang and J. Zhu. Portfolio theory of information retrieval. In SIGIR'09, 115--122. ACM, 2009.

Digital Library

[17]

J. Wang and J. Zhu. On statistical analysis and optimization of information retrieval effectiveness metrics. In SIGIR'10, 226--233, ACM, 2010.

Digital Library

[18]

Y. Yue and T. Joachims. Predicting diverse subsets using structural SVMs. In ICML'08, 1224--1231, ACM, 2008.

Digital Library

[19]

C. Zhai, W. W. Cohen, and J. Lafferty. Beyond independent relevance: Methods and evaluation metrics for subtopic retrieval. In SIGIR'03, pages 10--17. ACM, 2003.

Digital Library

Cited By

Duan YJatowt ATanaka K(2019)Discovering Latent Threads in Entity HistoriesData Science and Engineering10.1007/s41019-019-00108-x4:4(336-351)Online publication date: 15-Nov-2019
https://doi.org/10.1007/s41019-019-00108-x
Duan YJatowt ATanaka K(2019)History-Driven Entity CategorizationWeb and Big Data10.1007/978-3-030-26075-0_27(349-364)Online publication date: 17-Jul-2019
https://doi.org/10.1007/978-3-030-26075-0_27
Karako CManggala PMitrovic TZhang JChen LChin D(2018)Using Image Fairness Representations in Diversity-Based Re-ranking for RecommendationsAdjunct Publication of the 26th Conference on User Modeling, Adaptation and Personalization10.1145/3213586.3226206(23-28)Online publication date: 2-Jul-2018
https://dl.acm.org/doi/10.1145/3213586.3226206
Show More Cited By

Index Terms

Diverse retrieval via greedy optimization of expected 1-call@k in a latent subtopic relevance model
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Probabilistic latent maximal marginal relevance
SIGIR '10: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval

Diversity has been heavily motivated in the information retrieval literature as an objective criterion for result sets in search and recommender systems. Perhaps one of the most well-known and most used algorithms for result set diversification is that ...
Beyond independent relevance: methods and evaluation metrics for subtopic retrieval
SIGIR '03: Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval

We present a non-traditional retrieval problem we call subtopic retrieval. The subtopic retrieval problem is concerned with finding documents that cover many different subtopics of a query topic. In such a problem, the utility of a document in a ranking ...
Beyond Independent Relevance: Methods and Evaluation Metrics for Subtopic Retrieval

We present a non-traditional retrieval problem we call subtopic retrieval. The subtopic retrieval problem is concerned with finding documents that cover many different subtopics of a query topic. In such a problem, the utility of a document in a ranking ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management

October 2011

2712 pages

ISBN:9781450307178

DOI:10.1145/2063576

Editors:
Bettina Berendt,
Arjen de Vries,
Wenfei Fan,
Craig Macdonald
University of Glasgow, UK
,
Iadh Ounis
University of Glasgow, UK
,
Ian Ruthven
University of Strathclyde, UK

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 October 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

CIKM '11

Sponsor:

CIKM '11: International Conference on Information and Knowledge Management

October 24 - 28, 2011

Glasgow, Scotland, UK

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
168
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)1

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Duan YJatowt ATanaka K(2019)Discovering Latent Threads in Entity HistoriesData Science and Engineering10.1007/s41019-019-00108-x4:4(336-351)Online publication date: 15-Nov-2019
https://doi.org/10.1007/s41019-019-00108-x
Duan YJatowt ATanaka K(2019)History-Driven Entity CategorizationWeb and Big Data10.1007/978-3-030-26075-0_27(349-364)Online publication date: 17-Jul-2019
https://doi.org/10.1007/978-3-030-26075-0_27
Karako CManggala PMitrovic TZhang JChen LChin D(2018)Using Image Fairness Representations in Diversity-Based Re-ranking for RecommendationsAdjunct Publication of the 26th Conference on User Modeling, Adaptation and Personalization10.1145/3213586.3226206(23-28)Online publication date: 2-Jul-2018
https://dl.acm.org/doi/10.1145/3213586.3226206
Yu HJatowt ABlanco RJoho HJose JChen LYuan Fde Rijke MShokouhi MTomkins AZhang M(2017)A Concise Integer Linear Programming Formulation for Implicit Search Result DiversificationProceedings of the Tenth ACM International Conference on Web Search and Data Mining10.1145/3018661.3018710(191-200)Online publication date: 2-Feb-2017
https://dl.acm.org/doi/10.1145/3018661.3018710
Rao VJain PJawahar CKender JSmith JLuo JBoll SHsu W(2016)Diverse Yet Efficient Retrieval using Locality Sensitive HashingProceedings of the 2016 ACM on International Conference on Multimedia Retrieval10.1145/2911996.2911998(189-196)Online publication date: 6-Jun-2016
https://dl.acm.org/doi/10.1145/2911996.2911998
Kharazmi SScholer FVallet DSanderson M(2016)Examining Additivity and Weak BaselinesACM Transactions on Information Systems10.1145/288278234:4(1-18)Online publication date: 9-Jun-2016
https://dl.acm.org/doi/10.1145/2882782
Yu HJatowt ABlanco RJoho HJose JChen LYuan F(2016)An In-Depth Study of Implicit Search Result DiversificationInformation Retrieval Technology10.1007/978-3-319-48051-0_29(342-348)Online publication date: 15-Oct-2016
https://doi.org/10.1007/978-3-319-48051-0_29
Choi AVan Den Broeck GDarwiche A(2015)Tractable learning for structured probability spacesProceedings of the 24th International Conference on Artificial Intelligence10.5555/2832581.2832649(2861-2868)Online publication date: 25-Jul-2015
https://dl.acm.org/doi/10.5555/2832581.2832649
Yu HRen FLi JWang XGarofalakis MSoboroff ISuel TWang M(2014)Search Result Diversification via Filling Up Multiple KnapsacksProceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management10.1145/2661829.2661933(609-618)Online publication date: 3-Nov-2014
https://dl.acm.org/doi/10.1145/2661829.2661933
Lim KSanner SGuo SHersh WCallan JMaarek YSanderson M(2012)On the mathematical relationship between expected n-call@k and the relevance vs. diversity trade-offProceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval10.1145/2348283.2348497(1117-1118)Online publication date: 12-Aug-2012
https://dl.acm.org/doi/10.1145/2348283.2348497

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents