research-article

The Effects of Vertical Rank and Border on Aggregated Search Coherence and Search Behavior

Authors:

Jaime Arguello,

Robert CapraAuthors Info & Claims

CIKM '14: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management

Pages 539 - 548

https://doi.org/10.1145/2661829.2661930

Published: 03 November 2014 Publication History

Abstract

Aggregated search is the task of blending results from different search services, or verticals, into a set of web search results. Aggregated search coherence is the extent to which results from different sources focus on similar senses of an ambiguous or underspecified query. Prior work investigated the "spill-over" effect between a set of blended vertical results and the web results. These studies found that users are more likely to interact with the web results when the vertical results are more consistent with the user's intended query-sense. We extend this prior work by investigating three new research questions: (1) Does the spill-over effect generalize across different verticals? (2) Does the vertical rank moderate the level of spill-over? and (3) Does the presence of a border around the vertical results moderate the level of spill-over? We investigate four different verticals (images, news, shopping, and video) and measure spill-over using interaction measures associated with varying levels of engagement with the web results (bookmarks, clicks, scrolls, and mouseovers). Results from a large-scale crowdsourced study suggest that: (1) The spill-over effect generalizes across verticals, but is stronger for some verticals than others, (2) Vertical rank has a stronger moderating effect for verticals with a mid-level of spill-over, and (3) Including a border around the vertical results has a subtle moderating effect for those verticals with a low level of spill-over.

References

[1]

J. Arguello and R. Capra. The effect of aggregated search coherence on search behavior. In CIKM, pages 1293--1302. ACM, 2012.

Digital Library

[2]

J. Arguello, R. Capra, and W.-C. Wu. Factors affecting aggregated search coherence and search behavior. In CIKM, pages 1989--1998. ACM, 2013.

Digital Library

[3]

J. Arguello, F. Diaz, and J. Callan. Learning to aggregate vertical results into web search results. In CIKM, pages 201--210. ACM, 2011.

Digital Library

[4]

J. Arguello, F. Diaz, J. Callan, and J.-F. Crespo. Sources of evidence for vertical selection. In SIGIR, pages 315--322. ACM, 2009.

Digital Library

[5]

J. Arguello, F. Diaz, and J.-F. Paiement. Vertical selection in the presence of unlabeled verticals. In SIGIR, pages 691--698. ACM, 2010.

Digital Library

[6]

P. Bailey, N. Craswell, R. W. White, L. Chen, A. Satyanarayana, and S. M. Tahaghoghi. Evaluating search systems using result page context. In IIiX, pages 105--114. ACM, 2010.

Digital Library

[7]

G. Buscher, S. T. Dumais, and E. Cutrell. The good, the bad, and the random: An eye-tracking study of ad quality in web search. In SIGIR, pages 42--49. ACM, 2010.

Digital Library

[8]

C. W. Cleverdon. The aslib cranfield research project on the comparative efficiency of indexing systems. Aslib Proceedings, 12(12):421--431, 1960.

[9]

E. Cutrell and Z. Guan. What are you looking for?: An eye-tracking study of information usage in web search. In CHI, pages 407--416. ACM, 2007.

Digital Library

[10]

F. Diaz. Integration of news content into web results. In WSDM, pages 182--191. ACM, 2009.

Digital Library

[11]

F. Diaz and J. Arguello. Adaptation of offline vertical selection predictions in the presence of user feedback. In SIGIR, pages 323--330. ACM, 2009.

Digital Library

[12]

S. T. Dumais, G. Buscher, and E. Cutrell. Individual differences in gaze patterns for web search. In IIiX, pages 185--194. ACM, 2010.

Digital Library

[13]

G. Hotchkiss. Q&A with Marissa Mayer. http://searchengineland.com/qa-with-marissa-mayer-google-vpsearch-products-user-experience-10370. Accessed: 2014-06-01.

[14]

A. Hughes, T. Wilkens, B. M. Wildemuth, and G. Marchionini. Text or pictures? an eyetracking study of how people view digital video surrogates. In CIVR, pages 271--280. Springer-Verlag, 2003.

Digital Library

[15]

T. Joachims, L. Granka, B. Pan, H. Hembrooke, and G. Gay. Accurately interpreting clickthrough data as implicit feedback. In SIGIR, pages 154--161. ACM, 2005.

Digital Library

[16]

S. Kalyanaraman and J. D. Ivory. Enhanced information scent, selective discounting, or consummate breakdown: The psychological effects of web-based search results. Media Psychology, 12(3):295--319, 2009.

[17]

G. Keppel and T. D. Wickens. Design and Analysis: A Researcher's Handbook. Prentice Hall, 3 edition, 1991.

[18]

K. Koffka. Principles of Gestalt psychology. Harcourt, New York, 1935.

[19]

X. Li, Y.-Y. Wang, and A. Acero. Learning query intent from regularized click graphs. In SIGIR, pages 339--346. ACM, 2008.

Digital Library

[20]

H. A. A. Maqbali, F. Scholer, J. Thom, and M. Wu. Evaluating the effectiveness of visual summaries for web search. In ADCS, pages 36--43, 2010.

[21]

S. E. Palmer. Common region: A new principle of perceptual grouping. Cognitive Psychology, 24(3):436--447, 1992.

[22]

B. Pan, H. Hembrooke, T. Joachims, L. Lorigo, G. Gay, and L. Granka. In google we trust: Users' decisions on rank, position, and relevance. Journal of Computer-Mediated Communication, 12(3):801--823, 2007.

[23]

A. K. Ponnuswami, K. Pattabiraman, D. Brand, and T. Kanungo. Model characterization curves for federated search using click-logs: predicting user engagement metrics for the span of feasible operating points. In WWW, pages 67--76. ACM, 2011.

Digital Library

[24]

A. K. Ponnuswami, K. Pattabiraman, Q. Wu, R. Gilad-Bachrach, and T. Kanungo. On composition of a federated web search result page: Using online users to provide pairwise preference for heterogeneous verticals. In WSDM, pages 715--724. ACM, 2011.

Digital Library

[25]

M. Sanderson. Ambiguous queries: test collections need more sense. In SIGIR, pages 499--506. ACM, 2008.

Digital Library

[26]

R. L. T. Santos, C. Macdonald, and I. Ounis. Aggregated search result diversification. In ITCIR, pages 250--261. Springer-Verlag, 2011.

Digital Library

[27]

F. Stentiford. Visual attention: low-level and high-level viewpoints. SPIE, 8436:84360L-84360L-9, 2012.

[28]

S. Sushmita, H. Joho, M. Lalmas, and R. Villa. Factors affecting click-through behavior in aggregated search interfaces. In CIKM, pages 519--528. ACM, 2010.

Digital Library

[29]

C. Ware. Information Visualization: Perception for Design. Morgan Kaufmann, San Francisco, CA, USA, 2000.

Digital Library

[30]

K. Zhou, R. Cummins, M. Lalmas, and J. M. Jose. Evaluating aggregated search pages. In SIGIR, pages 115--124. ACM, 2012.

Digital Library

Cited By

Shao YMao JLiu YZhang MMa S(2022)From linear to non-linear: investigating the effects of right-rail results on complex SERPsAdvances in Computational Intelligence10.1007/s43674-021-00028-22:1Online publication date: 10-Jan-2022
https://doi.org/10.1007/s43674-021-00028-2
Zhang RXie XMao JLiu YZhang MMa S(2021)Constructing a Comparison-based Click Model for Web SearchProceedings of the Web Conference 202110.1145/3442381.3449918(270-283)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3449918
Steichen BLowe R(2021)How do multilingual users search? An investigation of query and result list language choicesJournal of the Association for Information Science and Technology10.1002/asi.2444372:6(759-776)Online publication date: 14-May-2021
https://dl.acm.org/doi/10.1002/asi.24443
Show More Cited By

Index Terms

The Effects of Vertical Rank and Border on Aggregated Search Coherence and Search Behavior
1. Information systems
  1. Information retrieval

Recommendations

Factors affecting aggregated search coherence and search behavior
CIKM '13: Proceedings of the 22nd ACM international conference on Information & Knowledge Management

Aggregated search is the task of incorporating results from different search services, or verticals, into the web search results. Aggregated search coherence refers to the extent to which results from different sources focus on similar senses of a given ...
The Effects of Aggregated Search Coherence on Search Behavior

Aggregated search is the task of combining results from multiple independent search systems in a single Search Engine Results Page (SERP). Aggregated search coherence refers to the extent to which different sources on the SERP focus on similar senses of ...
The effect of aggregated search coherence on search behavior
CIKM '12: Proceedings of the 21st ACM international conference on Information and knowledge management

Aggregated search is the task of blending results from different specialized search services, or verticals, into the web search results. Aggregated search coherence refers to the degree to which results from different systems focus on similar senses of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '14: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management

November 2014

2152 pages

ISBN:9781450325981

DOI:10.1145/2661829

General Chairs:
Jianzhong Li
Harbin Inst. of Technology
,
X. Sean Wang
Fudan University
,
Program Chairs:
Minos Garofalakis
Technical University of Crete, Greece
,
Ian Soboroff
National Institute of Standards, USA
,
Torsten Suel
New York University, USA
,
Min Wang
Google Research, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '14

Sponsor:

CIKM '14: 2014 ACM Conference on Information and Knowledge Management

November 3 - 7, 2014

Shanghai, China

Acceptance Rates

CIKM '14 Paper Acceptance Rate 175 of 838 submissions, 21%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
193
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shao YMao JLiu YZhang MMa S(2022)From linear to non-linear: investigating the effects of right-rail results on complex SERPsAdvances in Computational Intelligence10.1007/s43674-021-00028-22:1Online publication date: 10-Jan-2022
https://doi.org/10.1007/s43674-021-00028-2
Zhang RXie XMao JLiu YZhang MMa S(2021)Constructing a Comparison-based Click Model for Web SearchProceedings of the Web Conference 202110.1145/3442381.3449918(270-283)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3449918
Steichen BLowe R(2021)How do multilingual users search? An investigation of query and result list language choicesJournal of the Association for Information Science and Technology10.1002/asi.2444372:6(759-776)Online publication date: 14-May-2021
https://dl.acm.org/doi/10.1002/asi.24443
Pothirattanachaikul SYamamoto TYamamoto YYoshikawa MSalter ABarber JGadiraju U(2020)Analyzing the Effects of "People also ask" on Search Behaviors and BeliefsProceedings of the 31st ACM Conference on Hypertext and Social Media10.1145/3372923.3404786(101-110)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.1145/3372923.3404786
Abualsaud MO'Brien HFreund LArapakis IHoeber OLopatovska I(2020)The Effect of Queries and Search Result Quality on the Rate of Query Abandonment in Interactive Information RetrievalProceedings of the 2020 Conference on Human Information Interaction and Retrieval10.1145/3343413.3377951(523-526)Online publication date: 14-Mar-2020
https://dl.acm.org/doi/10.1145/3343413.3377951
Arguello JChoi B(2019)The Effects of Working Memory, Perceptual Speed, and Inhibition in Aggregated SearchACM Transactions on Information Systems10.1145/332212837:3(1-34)Online publication date: 16-May-2019
https://dl.acm.org/doi/10.1145/3322128
Mackenzie JGupta KQiao FAwadallah AShokouhi M(2019)Exploring User Behavior in Email Re-Finding TasksThe World Wide Web Conference10.1145/3308558.3313450(1245-1255)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3308558.3313450
Levi OGuy IRaiber FKurland O(2018)Selective Cluster Presentation on the Search Results PageACM Transactions on Information Systems10.1145/315867236:3(1-42)Online publication date: 28-Feb-2018
https://dl.acm.org/doi/10.1145/3158672
Arguello J(2017)Aggregated SearchFoundations and Trends in Information Retrieval10.1561/150000005210:5(365-502)Online publication date: 6-Mar-2017
https://dl.acm.org/doi/10.1561/1500000052
Liu ZMao JWang CAi QLiu YNie J(2017)Enhancing click models with mouse movement informationInformation Retrieval10.1007/s10791-016-9292-420:1(53-80)Online publication date: 1-Feb-2017
https://dl.acm.org/doi/10.1007/s10791-016-9292-4
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents