Article

Predicting Search Task Difficulty

Author:

Jaime ArguelloAuthors Info & Claims

ECIR 2014: Proceedings of the 36th European Conference on IR Research on Advances in Information Retrieval - Volume 8416

Pages 88 - 99

https://doi.org/10.1007/978-3-319-06028-6_8

Published: 13 April 2014 Publication History

Abstract

Search task difficulty refers to a user's assessment about the amount of effort required to complete a search task. Our goal in this work is to learn predictive models of search task difficulty. We evaluate features derived from the user's interaction with the search engine as well as features derived from the user's level of interest in the task and level of prior knowledge in the task domain. In addition to user-interaction features used in prior work, we evaluate features generated from scroll and mouse-movement events on the SERP. In some situations, we may prefer a system that can predict search task difficulty early in the search session. To this end, we evaluate features in terms of whole-session evidence and first-round evidence, which excludes all interactions starting with the second query. Our results found that the most predictive features were different for whole-session vs.ï źfirst-round prediction, that mouseover features were effective for first-round prediction, and that level of interest and prior knowledge features did not improve performance.

References

[1]

Anderson, L.W., Krathwohl, D.R.: A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives 2001

[2]

Aula, A., Khan, R.M., Guan, Z.: How does search behavior change as search becomes more difficult? In: CHI, pp. 35---44 2010

Digital Library

[3]

Bell, D.J., Ruthven, I.: Searchers' assessments of task complexity for web searching. In: McDonald, S., Tait, J.I. eds. ECIR 2004. LNCS, vol. 2997, pp. 57---71. Springer, Heidelberg 2004

[4]

Byström, K., Järvelin, K.: Task complexity affects information seeking and use. Inf. Process. Manage. 312, 191---213 1995

Digital Library

[5]

Campbell, D.J.: Task complexity: A review and analysis. The Academy of Management Review 131, 40---52 1988

[6]

Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J.: Liblinear: A library for large linear classification. JMLR 9, 1871---1874 2008

Digital Library

[7]

Feild, H.A., Allan, J., Jones, R.: Predicting searcher frustration. In: SIGIR, pp. 34---41 2010

Digital Library

[8]

Jansen, B.J., Booth, D., Smith, B.: Using the taxonomy of cognitive learning to model online searching. Inf. Process. Manage. 456, 643---663 2009

Digital Library

[9]

Kim, J.: Task difficulty as a predictor and indicator of web searching interaction. In: CHI, pp. 959---964 2006

Digital Library

[10]

Li, Y., Belkin, N.J.: A faceted approach to conceptualizing tasks in information seeking. Inf. Process. Manage. 446, 1822---1837 2008

Digital Library

[11]

Liu, J., Cole, M.J., Liu, C., Bierig, R., Gwizdka, J., Belkin, N.J., Zhang, J., Zhang, X.: Search behaviors in different task types. In: JCDL, pp. 69---78 2010

Digital Library

[12]

Liu, J., Gwizdka, J., Liu, C., Belkin, N.J.: Predicting task difficulty for different task types. ASIS&T, 16:1---16:10 2010

Digital Library

[13]

Liu, J., Liu, C., Cole, M., Belkin, N.J., Zhang, X.: Exploring and predicting search task difficulty. In: CIKM, pp. 1313---1322 2012

Digital Library

[14]

Liu, J., Liu, C., Gwizdka, J., Belkin, N.J.: Can search systems detect users' task difficulty?: some behavioral signals. In: SIGIR, pp. 845---846 2010

Digital Library

[15]

Smucker, M.D., Allan, J., Carterette, B.: A comparison of statistical significance tests for information retrieval evaluation. In: CIKM, pp. 623---632 2007

Digital Library

[16]

White, R.W., Dumais, S.T.: Characterizing and predicting search engine switching behavior. In: CIKM, pp. 87---96 2009

Digital Library

[17]

Wood, R.E.: Task complexity: Definition of the construct. Organizational Behavior and Human Decision Processes 371, 60---82 1986

[18]

Wu, W.-C., Kelly, D., Edwards, A., Arguello, J.: Grannies, tanning beds, tattoos and nascar: evaluation of search tasks with varying levels of cognitive complexity. In: IIIX, pp. 254---257 2012

Digital Library

Cited By

Liu J(2021)Deconstructing search tasks in interactive information retrievalInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10252258:3Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1016/j.ipm.2021.102522
Huang KChen JLiu CZhang LHuang RWu DMarchionini GHe DCunningham SHansen P(2020)A Comparative Study of the Relationship between the Subjective Difficulty, Objective Difficulty of Search Tasks and Search BehaviorsProceedings of the ACM/IEEE Joint Conference on Digital Libraries in 202010.1145/3383583.3398614(421-424)Online publication date: 1-Aug-2020
https://dl.acm.org/doi/10.1145/3383583.3398614
Choi BWard ALi YArguello JCapra R(2019)The Effects of Task Complexity on the Use of Different Types of Information in a Search Assistance ToolACM Transactions on Information Systems10.1145/337170738:1(1-28)Online publication date: 20-Dec-2019
https://dl.acm.org/doi/10.1145/3371707
Show More Cited By

Predicting Search Task Difficulty
1. Information systems

Recommendations

Exploring and predicting search task difficulty
CIKM '12: Proceedings of the 21st ACM international conference on Information and knowledge management

We report on an investigation of behavioral differences between users in difficult and easy search tasks. Behavioral factors that can be used in real-time to predict task difficulty are identified. User data was collected in a controlled lab experiment (...
Predicting task difficulty for different task types
ASIS&T '10: Proceedings of the 73rd ASIS&T Annual Meeting on Navigating Streams in an Information Ecosystem - Volume 47

This paper reports our investigation of differences in users' behavior between difficult and easy search tasks, as well as how these differences vary with different types of tasks. We also report how behavioral predictors of task difficulty vary across ...
Predicting Search Task Difficulty at Different Search Stages
CIKM '14: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management

Knowing, in real time, whether a current searcher in an information retrieval system finds the search task difficult can be valuable for tailoring the system's support for that searcher. This study investigated searcher's behaviors at different stages of ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ECIR 2014: Proceedings of the 36th European Conference on IR Research on Advances in Information Retrieval - Volume 8416

April 2014

826 pages

ISBN:9783319060279

Editors:
Maarten Rijke
University of Amsterdam, Amsterdam, The Netherlands
,
Tom Kenter
University of Amsterdam, Amsterdam, The Netherlands
,
Arjen Vries
Centrum Wiskunde en Informatica, Amsterdam, The Netherlands and Delft University of Technology, Delft, The Netherlands
,
Chengxiang Zhai
University of Illinois at Urbana-Champaign, Urbana, USA
,
Franciska Jong
University of Twente, Twente, The Netheralnds and Erasmus University Rotterdam, Rotterdam, The Netherlands
,
Kira Radinsky
SalesPredict, Haifa, Israel
,
Katja Hofmann
Microsoft Research, Cambridge, UK

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 13 April 2014

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu J(2021)Deconstructing search tasks in interactive information retrievalInformation Processing and Management: an International Journal10.1016/j.ipm.2021.10252258:3Online publication date: 1-May-2021
https://dl.acm.org/doi/10.1016/j.ipm.2021.102522
Huang KChen JLiu CZhang LHuang RWu DMarchionini GHe DCunningham SHansen P(2020)A Comparative Study of the Relationship between the Subjective Difficulty, Objective Difficulty of Search Tasks and Search BehaviorsProceedings of the ACM/IEEE Joint Conference on Digital Libraries in 202010.1145/3383583.3398614(421-424)Online publication date: 1-Aug-2020
https://dl.acm.org/doi/10.1145/3383583.3398614
Choi BWard ALi YArguello JCapra R(2019)The Effects of Task Complexity on the Use of Different Types of Information in a Search Assistance ToolACM Transactions on Information Systems10.1145/337170738:1(1-28)Online publication date: 20-Dec-2019
https://dl.acm.org/doi/10.1145/3371707
Liu JShah CAzzopardi LHalvey MRuthven IJoho HMurdock VQvarfordt P(2019)Investigating the Impacts of Expectation Disconfirmation on Web SearchProceedings of the 2019 Conference on Human Information Interaction and Retrieval10.1145/3295750.3298959(319-323)Online publication date: 8-Mar-2019
https://dl.acm.org/doi/10.1145/3295750.3298959
Choi BCapra RArguello JAzzopardi LHalvey MRuthven IJoho HMurdock VQvarfordt P(2019)The Effects of Working Memory during Search Tasks of Varying ComplexityProceedings of the 2019 Conference on Human Information Interaction and Retrieval10.1145/3295750.3298948(261-265)Online publication date: 8-Mar-2019
https://dl.acm.org/doi/10.1145/3295750.3298948
Mitsui MLiu JShah CCollins-Thompson KMei QDavison BLiu YYilmaz E(2018)How Much is Too Much?The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval10.1145/3209978.3210105(1141-1144)Online publication date: 27-Jun-2018
https://dl.acm.org/doi/10.1145/3209978.3210105
Yu RGadiraju UHoltz PRokicki MKemkes PDietze SCollins-Thompson KMei QDavison BLiu YYilmaz E(2018)Predicting User Knowledge Gain in Informational Search SessionsThe 41st International ACM SIGIR Conference on Research & Development in Information Retrieval10.1145/3209978.3210064(75-84)Online publication date: 27-Jun-2018
https://dl.acm.org/doi/10.1145/3209978.3210064
Gadiraju UYu RDietze SHoltz PShah CBelkin NByström KHuang JScholer F(2018)Analyzing Knowledge Gain of Users in Informational Search Sessions on the WebProceedings of the 2018 Conference on Human Information Interaction & Retrieval10.1145/3176349.3176381(2-11)Online publication date: 1-Mar-2018
https://dl.acm.org/doi/10.1145/3176349.3176381
Keyaki AMiyazaki J(2018)Analysis of Relevant Text Fragments for Different Search Task TypesInformation Retrieval Technology10.1007/978-3-030-03520-4_6(60-66)Online publication date: 28-Nov-2018
https://dl.acm.org/doi/10.1007/978-3-030-03520-4_6
Jiang JHe DAllan JKando NSakai TJoho HLi Hde Vries AWhite R(2017)Comparing In Situ and Multidimensional Relevance JudgmentsProceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3077136.3080840(405-414)Online publication date: 7-Aug-2017
https://dl.acm.org/doi/10.1145/3077136.3080840
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents