short-paper

Query Performance Prediction for Pseudo-Feedback-Based Retrieval

Authors:

Haggai Roitman,

Oren KurlandAuthors Info & Claims

SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 1261 - 1264

https://doi.org/10.1145/3331184.3331369

Published: 18 July 2019 Publication History

Abstract

The query performance prediction task (QPP) is estimating retrieval effectiveness in the absence of relevance judgments. Prior work has focused on prediction for retrieval methods based on surface level query-document similarities (e.g., query likelihood). We address the prediction challenge for pseudo-feedback-based retrieval methods which utilize an initial retrieval to induce a new query model; the query model is then used for a second (final) retrieval. Our suggested approach accounts for the presumed effectiveness of the initially retrieved list, its similarity with the final retrieved list and properties of the latter. Empirical evaluation demonstrates the clear merits of our approach.

References

[1]

Giambattista Amati, Claudio Carpineto, and Giovanni Romano. Query difficulty, robustness, and selective application of query expansion. In Proceedings of ECIR, pages 127--137, 2004.

[2]

David Carmel and Oren Kurland. Query performance prediction for ir. In Proceedings of SIGIR, pages 1196--1197, New York, NY, USA, 2012. ACM.

Digital Library

[3]

Claudio Carpineto and Giovanni Romano. A survey of automatic query expansion in information retrieval. ACM Comput. Surv., 44(1):1:1--1:50, January 2012.

Digital Library

[4]

Steve Cronen-Townsend, Yun Zhou, and W. Bruce Croft. Predicting query performance. In Proceedings of SIGIR, pages 299--306, New York, NY, USA, 2002. ACM.

Digital Library

[5]

Steve Cronen-Townsend, Yun Zhou, and W. Bruce Croft. A framework for selective query expansion. In Proceedings of CIKM, pages 236--237, New York, NY, USA, 2004. ACM.

Digital Library

[6]

Ahmad Khwileh, Andy Way, and Gareth J. F. Jones. Improving the reliability of query expansion for user-generated speech retrieval using query performance prediction. In CLEF, 2017.

[7]

Oren Kurland, Anna Shtok, Shay Hummel, Fiana Raiber, David Carmel, and Ofri Rom. Back to the roots: A probabilistic framework for query-performance prediction. In Proceedings of CIKM, pages 823--832, New York, NY, USA, 2012. ACM.

Digital Library

[8]

Victor Lavrenko and W. Bruce Croft. Relevance based language models. In Proceedings of SIGIR.

Digital Library

[9]

Yuanhua Lv and ChengXiang Zhai. Revisiting the divergence minimization feedback model. In CIKM '14.

Digital Library

[10]

Fiana Raiber and Oren Kurland. Query-performance prediction: setting the expectations straight. In Proceedings of SIGIR, pages 13--22, 2014.

Digital Library

[11]

Fiana Raiber, Oren Kurland, Filip Radlinski, and Milad Shokouhi. Learning asymmetric co-relevance. In Proceedings of ICTIR, pages 281--290, 2015.

Digital Library

[12]

Haggai Roitman. Enhanced performance prediction of fusion-based retrieval. In Proceedings of ICTIR, pages 195--198, New York, NY, USA, 2018. ACM.

Digital Library

[13]

Haggai Roitman. An extended query performance prediction framework utilizing passage-level information. In Proceedings of ICTIR, pages 35--42, New York, NY, USA, 2018. ACM.

Digital Library

[14]

Haggai Roitman, Ella Rabinovich, and Oren Sar Shalom. As stable as you are: Re-ranking search results using query-drift analysis. In Proceedings of HT, pages 33--37, New York, NY, USA, 2018. ACM.

Digital Library

[15]

Harrisen Scells, Leif Azzopardi, Guido Zuccon, and Bevan Koopman. Query variation performance prediction for systematic reviews. In Proceedings of SIGIR, pages 1089--1092, New York, NY, USA, 2018. ACM.

Digital Library

[16]

Anna Shtok, Oren Kurland, and David Carmel. Using statistical decision theory and relevance models for query-performance prediction. In Proceedings of SIGIR, pages 259--266, New York, NY, USA, 2010. ACM.

Digital Library

[17]

Anna Shtok, Oren Kurland, and David Carmel. Query performance prediction using reference lists. ACM Trans. Inf. Syst., 34(4):19:1--19:34, June 2016.

Digital Library

[18]

Anna Shtok, Oren Kurland, David Carmel, Fiana Raiber, and Gad Markovits. Predicting query performance by query-drift estimation. ACM Trans. Inf. Syst., 30(2):11:1--11:35, May 2012.

Digital Library

[19]

William Webber, Alistair Moffat, and Justin Zobel. A similarity measure for indefinite rankings. ACM Trans. Inf. Syst., 28(4):20:1--20:38, November 2010.

Digital Library

[20]

Chengxiang Zhai and John Lafferty. Model-based feedback in the language modeling approach to information retrieval. In CIKM '01.

Digital Library

[21]

Yun Zhou and W. Bruce Croft. Query performance prediction in web search environments. In Proceedings of SIGIR, pages 543--550, New York, NY, USA, 2007. ACM.

Digital Library

Cited By

Arabzadeh NHamidi Rad RKhodabakhsh MBagheri EFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Noisy Perturbations for Estimating Query Difficulty in Dense RetrieversProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615270(3722-3727)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615270
Poesina EIonescu RMothe JChen HDuh WHuang HKato MMothe JPoblete B(2023)iQPP: A Benchmark for Image Query Performance PredictionProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591901(2953-2963)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591901
Datta SGanguly DMitra MGreene D(2022)A Relative Information Gain-based Query Performance Prediction Framework with Generated Query VariantsACM Transactions on Information Systems10.1145/354511241:2(1-31)Online publication date: 23-Jun-2022
https://dl.acm.org/doi/10.1145/3545112
Show More Cited By

Index Terms

Query Performance Prediction for Pseudo-Feedback-Based Retrieval
1. Information systems
  1. Information retrieval

Recommendations

Query dependent pseudo-relevance feedback based on wikipedia
SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval

Pseudo-relevance feedback (PRF) via query-expansion has been proven to be e®ective in many information retrieval (IR) tasks. In most existing work, the top-ranked documents from an initial search are assumed to be relevant and used for PRF. One problem ...
Information retrieval with concept-based pseudo-relevance feedback in MEDLINE

Although using domain specific knowledge sources for information retrieval yields more accurate results compared to pure keyword-based methods, more improvements can be achieved by considering both relations between concepts in an ontology and also their ...
Information Retrieval Based on Pseudo Prediction Query Performance
ETCS '09: Proceedings of the 2009 First International Workshop on Education Technology and Computer Science - Volume 02

Prediction query performance (PQP) has recently been recognized by the IR (information retrieval) community as an important capability for IR systems. Put forward pseudo prediction query performance in the base of analyzing PQP.Design a new IR method. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2019

1512 pages

ISBN:9781450361729

DOI:10.1145/3331184

General Chairs:
Benjamin Piwowarski
CNRS - Sorbonne Universite, France
,
Max Chevalier
Universite de Toulouse, CNRS, France
,
Eric Gaussier
Universite Grenoble Alpes, CNRS, France
,
Program Chairs:
Yoelle Maarek
Amazon Research, Israel
,
Jian-Yun Nie
University of Montreal, Canada
,
Falk Scholer
RMIT University, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Short-paper

Funding Sources

Israel Science Foundation

Conference

SIGIR '19

Sponsor:

SIGIR

SIGIR '19: The 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval

July 21 - 25, 2019

Paris, France

Acceptance Rates

SIGIR'19 Paper Acceptance Rate 84 of 426 submissions, 20%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
219
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Arabzadeh NHamidi Rad RKhodabakhsh MBagheri EFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Noisy Perturbations for Estimating Query Difficulty in Dense RetrieversProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615270(3722-3727)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615270
Poesina EIonescu RMothe JChen HDuh WHuang HKato MMothe JPoblete B(2023)iQPP: A Benchmark for Image Query Performance PredictionProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3539618.3591901(2953-2963)Online publication date: 19-Jul-2023
https://dl.acm.org/doi/10.1145/3539618.3591901
Datta SGanguly DMitra MGreene D(2022)A Relative Information Gain-based Query Performance Prediction Framework with Generated Query VariantsACM Transactions on Information Systems10.1145/354511241:2(1-31)Online publication date: 23-Jun-2022
https://dl.acm.org/doi/10.1145/3545112
Huang YXu QLiu YXu CWu S(2022)Data Fusion Methods with Graded Relevance JudgmentWeb Information Systems and Applications10.1007/978-3-031-20309-1_20(227-239)Online publication date: 8-Dec-2022
https://doi.org/10.1007/978-3-031-20309-1_20
Chen XHe BSun L(2022)Groupwise Query Performance Prediction with BERTAdvances in Information Retrieval10.1007/978-3-030-99739-7_8(64-74)Online publication date: 10-Apr-2022
https://dl.acm.org/doi/10.1007/978-3-030-99739-7_8
Roitman HBalog KSetty VLioma CLiu YZhang MBerberich K(2020)ICTIR Tutorial: Modern Query Performance Prediction: Theory and PracticeProceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval10.1145/3409256.3409813(195-196)Online publication date: 14-Sep-2020
https://dl.acm.org/doi/10.1145/3409256.3409813

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents