Article

Space-Limited ranked query evaluation using adaptive pruning

Authors:

Nicholas Lester,

Alistair Moffat,

William Webber,

Justin ZobelAuthors Info & Claims

WISE'05: Proceedings of the 6th international conference on Web Information Systems Engineering

Pages 470 - 477

https://doi.org/10.1007/11581062_37

Published: 20 November 2005 Publication History

Publisher Site

Abstract

Evaluation of ranked queries on large text collections can be costly in terms of processing time and memory space. Dynamic pruning techniques allow both costs to be reduced, at the potential risk of decreased retrieval effectiveness. In this paper we describe an improved query pruning mechanism that offers a more resilient tradeoff between query evaluation costs and retrieval effectiveness than do previous pruning approaches.

References

[1]

V. N. Anh, O. de Kretser, and A. Moffat. Vector-space ranking with effective early termination. InW. B. Croft, D. J. Harper, D. H. Kraft, and J. Zobel, editors, Proc. 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 35-42, New Orleans, LA, Sept. 2001. ACM Press, New York.

Crossref

Google Scholar

[2]

L. A. Barroso, J. Dean, and U. Hölzle. Web search for a planet: The Google cluster architecture. IEEE Micro, 23(2):22-28, Mar. 2003.

Crossref

Google Scholar

[3]

C. Buckley and A. F. Lewit. Optimization of inverted vector searches. In Proc. 8th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 97-110, Montreal, Canada, June 1985. ACM Press, New York.

Crossref

Google Scholar

[4]

C. Buckley and E. M. Voorhees. Evaluating evaluation measure stability. In N. J. Belkin, P. Ingwersen, and M.-K. Leong, editors, Proc. 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 33-40, Athens, Greece, Sept. 2000. ACM Press, New York.

Crossref

Google Scholar

[5]

D. K. Harman and G. Candela. Retrieving records from a gigabyte of text on a minicomputer using statistical ranking. Journal of the American Society for Information Science, 41(8): 581-589, Aug. 1990.

Google Scholar

[6]

A. Moffat and J. Zobel. Self-indexing inverted files for fast text retrieval. ACM Transactions on Information Systems, 14(4):349-379, Oct. 1996.

Crossref

Google Scholar

[7]

M. Persin, J. Zobel, and R. Sacks-Davis. Filtered document retrieval with frequency-sorted indexes. Journal of the American Society for Information Science, 47(10):749-764, Oct. 1996.

Crossref

Google Scholar

[8]

A. Smeaton and C. J. van Rijsbergen. The nearest neighbour problem in information retrieval. In C. J. Crouch, editor, Proc. 4th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 83-87, Oakland, California, May 1981. ACM Press, New York.

Crossref

Google Scholar

[9]

I. H. Witten, A. Moffat, and T. C. Bell. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann, San Francisco, second edition, 1999.

Crossref

Google Scholar

[10]

W. Y. P. Wong and D. K. Lee. Implementations of partial document ranking using inverted files. Information Processing & Management, 29(5):647-669, Sept. 1993.

Crossref

Google Scholar

[11]

C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to information retrieval. ACM Transactions on Information Systems, 22(2):179-214, Apr. 2004.

Crossref

Google Scholar

Cited By

View all

Fontoura MJosifovski VLiu JVenkatesan SZhu XZien J(2020)Evaluation strategies for top-k queries over memory-resident inverted indexesProceedings of the VLDB Endowment10.14778/3402755.34027564:12(1213-1224)Online publication date: 3-Jun-2020
https://dl.acm.org/doi/10.14778/3402755.3402756
Broder AGarcia-Pueyo LJosifovski VVassilvitskii SVenkatesan SCarterette BDiaz FCastillo CMetzler D(2014)Scalable K-Means by ranked retrievalProceedings of the 7th ACM international conference on Web search and data mining10.1145/2556195.2556260(233-242)Online publication date: 24-Feb-2014
https://dl.acm.org/doi/10.1145/2556195.2556260
Broccolo DMacdonald COrlando SOunis IPerego RSilvestri FTonellotto NHe QIyengar ANejdl WPei JRastogi R(2013)Load-sensitive selective pruning for distributed searchProceedings of the 22nd ACM international conference on Information & Knowledge Management10.1145/2505515.2505699(379-388)Online publication date: 27-Oct-2013
https://dl.acm.org/doi/10.1145/2505515.2505699
Show More Cited By

Space-Limited ranked query evaluation using adaptive pruning
1. Information systems

Recommendations

On Using Query Logs for Static Index Pruning
WI-IAT '10: Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 01

Static index pruning techniques aim at removing from the posting lists of an inverted file the references to documents which are likely to be not relevant for answering user queries. The reduction in the size of the index results in a better ...
Query evaluation using overlapping views: completeness and efficiency
SIGMOD '06: Proceedings of the 2006 ACM SIGMOD international conference on Management of data

We study the problem of finding efficient equivalent view-based rewritings of relational queries, focusing on query optimization using materialized views under the assumption that base relations cannot contain duplicate tuples. A lot of work in the ...
Answering Queries Using Limited External Query Processors

When answering queries using external information sources, the contents of the queries can be described by views. To answer a query, we must rewrite it using the set of views presented by the sources. When the external information sources also have the ...

Comments

Information & Contributors

Information

Published In

WISE'05: Proceedings of the 6th international conference on Web Information Systems Engineering

November 2005

767 pages

ISBN:3540300171

Editors:
Anne H. Ngu
Texas State University, San Marcos, TX
,
Masaru Kitsuregawa
Institute of Industrial Science, The University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo, TX, Japan
,
Erich J. Neuhold
Institute of Industrial Science, University of Vienna, 4-6-1 Komaba, Meguro-ku, Vienna, TX, Austria
,
Jen-Yao Chung
IBM Research Division, Thomas J. Watson Research Center, 4-6-1 Komaba, Meguro-ku, New York, Yorktown Heights, TX, Austria
,
Quan Z. Sheng
School of Computer Science and Engineering, University of New South Wales, 4-6-1 Komaba, Meguro-ku, Sydney, TX, Australia

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 20 November 2005

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Fontoura MJosifovski VLiu JVenkatesan SZhu XZien J(2020)Evaluation strategies for top-k queries over memory-resident inverted indexesProceedings of the VLDB Endowment10.14778/3402755.34027564:12(1213-1224)Online publication date: 3-Jun-2020
https://dl.acm.org/doi/10.14778/3402755.3402756
Broder AGarcia-Pueyo LJosifovski VVassilvitskii SVenkatesan SCarterette BDiaz FCastillo CMetzler D(2014)Scalable K-Means by ranked retrievalProceedings of the 7th ACM international conference on Web search and data mining10.1145/2556195.2556260(233-242)Online publication date: 24-Feb-2014
https://dl.acm.org/doi/10.1145/2556195.2556260
Broccolo DMacdonald COrlando SOunis IPerego RSilvestri FTonellotto NHe QIyengar ANejdl WPei JRastogi R(2013)Load-sensitive selective pruning for distributed searchProceedings of the 22nd ACM international conference on Information & Knowledge Management10.1145/2505515.2505699(379-388)Online publication date: 27-Oct-2013
https://dl.acm.org/doi/10.1145/2505515.2505699
Jonassen SMille AGandon FMisselis JRabinovich MStaab S(2012)Scalable search platformProceedings of the 21st International Conference on World Wide Web10.1145/2187980.2187999(145-150)Online publication date: 16-Apr-2012
https://dl.acm.org/doi/10.1145/2187980.2187999
Jonassen SBratsberg S(2012)Improving the performance of pipelined query processing with skippingProceedings of the 13th international conference on Web Information Systems Engineering10.1007/978-3-642-35063-4_1(1-15)Online publication date: 28-Nov-2012
https://dl.acm.org/doi/10.1007/978-3-642-35063-4_1
Jonassen SBratsberg S(2011)Efficient compressed inverted index skipping for disjunctive text-queriesProceedings of the 33rd European conference on Advances in information retrieval10.5555/1996889.1996958(530-542)Online publication date: 18-Apr-2011
https://dl.acm.org/doi/10.5555/1996889.1996958
Ozcan RAltingovde IUlusoy Ö(2011)Cost-Aware Strategies for Query Result Caching in Web Search EnginesACM Transactions on the Web10.1145/1961659.19616635:2(1-25)Online publication date: 1-May-2011
https://dl.acm.org/doi/10.1145/1961659.1961663
Jonassen SBratsberg S(2011)Efficient Compressed Inverted Index Skipping for Disjunctive Text-QueriesProceedings of the 33rd European Conference on Advances in Information Retrieval - Volume 661110.1007/978-3-642-20161-5_53(530-542)Online publication date: 18-Apr-2011
https://dl.acm.org/doi/10.1007/978-3-642-20161-5_53
Jonassen SBratsberg S(2010)A combined semi-pipelined query processing architecture for distributed full-text retrievalProceedings of the 11th international conference on Web information systems engineering10.5555/1991336.1991400(587-601)Online publication date: 12-Dec-2010
https://dl.acm.org/doi/10.5555/1991336.1991400
Webber WMoffat AZobel J(2010)A similarity measure for indefinite rankingsACM Transactions on Information Systems10.1145/1852102.185210628:4(1-38)Online publication date: 23-Nov-2010
https://dl.acm.org/doi/10.1145/1852102.1852106
Show More Cited By

Abstract

References

Cited By

Recommendations

On Using Query Logs for Static Index Pruning

Query evaluation using overlapping views: completeness and efficiency

Answering Queries Using Limited External Query Processors

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations