research-article

Open access

This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics

Authors:

Ujwal Gadiraju,

Alessandro Bozzon,

Benjamin TimmermansAuthors Info & Claims

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 295 - 305

https://doi.org/10.1145/3404835.3462851

Published: 11 July 2021 Publication History

Abstract

In web search on debated topics, algorithmic and cognitive biases strongly influence how users consume and process information. Recent research has shown that this can lead to a search engine manipulation effect (SEME): when search result rankings are biased towards a particular viewpoint, users tend to adopt this favored viewpoint. To better understand the mechanisms underlying SEME, we present a pre-registered, 5 x 3 factorial user study investigating whether order effects (i.e., users adopting the viewpoint pertaining to higher-ranked documents) can cause SEME. For five different debated topics, we evaluated attitude change after exposing participants with mild pre-existing attitudes to search results that were overall viewpoint-balanced but reflected one of three levels of algorithmic ranking bias. We found that attitude change did not differ across levels of ranking bias and did not vary based on individual user differences. Our results thus suggest that order effects may not be an underlying mechanism of SEME. Exploratory analyses lend support to the presence of exposure effects (i.e., users adopting the majority viewpoint among the results they examine) as a contributing factor to users' attitude change. We discuss how our findings can inform the design of user bias mitigation strategies.

Supplementary Material

MP4 File (fp0224.mp4)

Video presentation for the paper "This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics".

Download
43.14 MB

References

[1]

Adnan Abid, Naveed Hussain, Kamran Abid, Farooq Ahmad, Muhammad Shoaib Farooq, Uzma Farooq, Sher Afzal Khan, Yaser Daanial Khan, Muhammad Azhar Naeem, and Nabeel Sabir. 2015. A survey on search results diversification techniques. Neural Computing and Applications, Vol. 27, 5 (2015), 1207--1229. https://doi.org/10.1007/s00521-015--1945--5

Digital Library

[2]

Rakesh Agrawal, Sreenivas Gollapudi, Alan Halverson, and Samuel Ieong. 2009. Diversifying search results. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining, WSDM'09. Association for Computing Machinery, New York, NY, USA, 5--14. https://doi.org/10.1145/1498759.1498766

Digital Library

[3]

Ahmed Allam, Peter Johannes Schulz, and Kent Nakamoto. 2014. The impact of search engine selection and sorting criteria on vaccination beliefs and attitudes:Two experiments manipulating google output. Journal of Medical Internet Research, Vol. 16, 4 (2014), e100. https://doi.org/10.2196/jmir.2642

[4]

Amazon. [n.d.]. Amazon Mechanical Turk. https://www.mturk.com/

[5]

Abolfazl Asudeh, H. V. Jagadish, Julia Stoyanovich, and Gautam Das. 2019. Designing fair ranking schemes. In Proceedings of the ACM SIGMOD International Conference on Management of Data. Association for Computing Machinery, New York, NY, USA, 1259--1276. https://doi.org/10.1145/3299869.3300079 arxiv: 1712.09752

Digital Library

[6]

Leif Azzopardi. 2021. Cognitive Biases in Search: A Review and Reflection of Cognitive Biases in Information Retrieval. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval (CHIIR '21). Association for Computing Machinery, New York, NY, USA, 27--37. https://doi.org/10.1145/3406522.3446023

Digital Library

[7]

Ricardo Baeza-Yates. 2018. Bias on the web. Commun. ACM, Vol. 61, 6 (2018), 54--61. https://doi.org/10.1145/3209581

Digital Library

[8]

Asia J. Biega, Krishna P. Gummadi, and Gerhard Weikum. 2018. Equity of attention: Amortizing individual fairness in rankings. In 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2018. Association for Computing Machinery, New York, NY, USA, 405--414. https://doi.org/10.1145/3209978.3210063 arxiv: 1805.01788

Digital Library

[9]

Noel Carroll. 2014. In Search We Trust. International Journal of Knowledge Society Research, Vol. 5, 1 (2014), 12--27. https://doi.org/10.4018/ijksr.2014010102

Digital Library

[10]

L. Elisa Celis, Damian Straszak, and Nisheeth K. Vishnoi. 2018. Ranking with fairness constraints. Leibniz International Proceedings in Informatics, LIPIcs, Vol. 107 (2018), 1--32. https://doi.org/10.4230/LIPIcs.ICALP.2018.28 arxiv: arXiv:1704.06840v4

[11]

Jon Chamberlain, Udo Kruschwitz, and Orland Hoeber. 2019. Scalable visualisation of sentiment and stance. In LREC 2018 - 11th International Conference on Language Resources and Evaluation. 4181--4185.

[12]

Tong Chen, Hongzhi Yin, Guanhua Ye, Zi Huang, Yang Wang, and Meng Wang. 2020. Try this instead: Personalized and interpretable substitute recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 891--900.

Digital Library

[13]

Charles L.A. Clarke, Maheedhar Kolla, Gordon V. Cormack, Olga Vechtomova, Azin Ashkan, Stefan Bü ttcher, and Ian MacKinnon. 2008. Novelty and diversity in information retrieval evaluation. In ACM SIGIR 2008 - 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings. Association for Computing Machinery, New York, NY, USA, 659--666. https://doi.org/10.1145/1390334.1390446

Digital Library

[14]

Tim Draws, Nava Tintarev, Ujwal Gadiraju, Alessandro Bozzon, and Benjamin Timmermans. 2020. Assessing Viewpoint Diversity in Search Results Using Ranking Fairness Metrics. In Informal Proceedings of the Bias and Fairness in AI Workshop at ECML-PKDD (BIAS 2020) .

[15]

Robert Epstein and Ronald E. Robertson. 2015. The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the National Academy of Sciences of the United States of America, Vol. 112, 33 (2015), E4512--E4521. https://doi.org/10.1073/pnas.1419828112

[16]

Robert Epstein, Ronald E. Robertson, David Lazer, and Christo Wilson. 2017. Suppressing the Search Engine Manipulation Effect (SEME). Proceedings of the ACM on Human-Computer Interaction, Vol. 1, CSCW (2017), 1--22. https://doi.org/10.1145/3134677

Digital Library

[17]

Franz Faul, Edgar Erdfelder, Albert Georg Lang, and Axel Buchner. 2007. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, Vol. 39, 2 (2007), 175---191. https://doi.org/10.3758/BF03193146

[18]

Ruoyuan Gao and Chirag Shah. 2020. Toward creating a fairer ranking in search engine results. Information Processing and Management, Vol. 57, 1 (2020), 102138. https://doi.org/10.1016/j.ipm.2019.102138

Digital Library

[19]

Amira Ghenai, Mark D. Smucker, and Charles L.A. Clarke. 2020. A think-aloud study to understand factors affecting online health search. In CHIIR 2020 - Proceedings of the 2020 Conference on Human Information Interaction and Retrieval. 273--282. https://doi.org/10.1145/3343413.3377961

Digital Library

[20]

Danah Golebiewski, Michael and Boyd. 2019. Data voids: Where missing data can easily be exploited. Data & Society (2019).

[21]

Sander Greenland, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, and Douglas G. Altman. 2016. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. European Journal of Epidemiology, Vol. 31, 4 (2016), 337--350. https://doi.org/10.1007/s10654-016-0149--3

[22]

Alexander Haas and Julian Unkel. 2017. Ranking versus reputation: perception and effects of search result credibility. Behaviour and Information Technology, Vol. 36, 12 (2017), 1285--1298. https://doi.org/10.1080/0144929X.2017.1381166

Digital Library

[23]

Uriel Haran, Ilana Ritov, and Barbara A. Mellers. 2013. The role of actively open-minded thinking in information acquisition, accuracy, and calibration. Judgment and Decision Making, Vol. 8, 3 (2013), 188--201.

[24]

Robin M. Hogarth and Hillel J. Einhorn. 1992. Order effects in belief updating: The belief-adjustment model. Cognitive Psychology, Vol. 24, 1 (1992), 1--55. https://doi.org/10.1016/0010-0285(92)90002-J

[25]

JASP_Team. 2020. JASP (Version 0.14) .

[26]

Harold Jeffreys. 1939. Theory of Probability. Oxford University Press, Oxford.

[27]

Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2005. Accurately interpreting clickthrough data as implicit feedback. SIGIR 2005 - Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Vol. 51, 1 (2005), 154--161. https://doi.org/10.1145/1076034.1076063

Digital Library

[28]

Yvonne Kammerer and Peter Gerjets. 2012. How Search Engine Users Evaluate and Select Web Search Results: The Impact of the Search Engine Interface on Credibility Assessments. In Library and Information Science, D. Lewandowski (Ed.). Vol. 12. Emerald Group Publishing Ltd, Bingley, Chapter 10, 251--279. https://doi.org/10.1108/s1876-0562(2012)002012a012

[29]

Mark T Keane and Maeve O'Brien. 2008. Are people biased in their use of search engines? Commun. ACM, Vol. 51, 2 (2008), 49--52.

Digital Library

[30]

Bart P. Knijnenburg, Martijn C. Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell. 2012. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction, Vol. 22, 4--5 (2012), 441--504. https://doi.org/10.1007/s11257-011--9118--4

Digital Library

[31]

Silvia Knobloch-Westerwick, Benjamin K. Johnson, and Axel Westerwick. 2015. Confirmation bias in online searches: Impacts of selective exposure before an election on political attitude strength and shifts. Journal of Computer-Mediated Communication, Vol. 20, 2 (2015), 171--187. https://doi.org/10.1111/jcc4.12105

[32]

Klaus Krippendorff. 2004. Reliability in Content Analysis. Human Communication Research, Vol. 30, 3 (2004), 411--433. https://doi.org/10.1111/j.1468--2958.2004.tb00738.x

[33]

Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics, Vol. 22, 1 (1951), 79--86.

[34]

Juhi Kulshrestha, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P. Gummadi, and Karrie Karahalios. 2019. Search bias quantification: investigating political bias in social media and web search. Information Retrieval Journal, Vol. 22, 1--2 (2019), 188--227. https://doi.org/10.1007/s10791-018--9341--2

Digital Library

[35]

Michael D. Lee and Eric Jan Wagenmakers. 2014. Bayesian cognitive modeling: A practical course. Cambridge University Press. https://doi.org/10.1017/CBO9781139087759

[36]

Ramona Ludolph, Ahmed Allam, and Peter J. Schulz. 2016. Manipulating Google's Knowledge Graph box to counter biased information processing during an online search on vaccination: Application of a technological debiasing strategy. Journal of Medical Internet Research, Vol. 18, 6 (2016), e137. https://doi.org/10.2196/jmir.5430

[37]

Microsoft. [n.d.]. Bing. https://www.bing.com/

[38]

Richard L. Miller. 1976. Mere Exposure, Psychological Reactance and Attitude Change. The Public Opinion Quarterly, Vol. 40, 2 (1976), 229--233.

[39]

Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. 2020. Controlling Fairness and Bias in Dynamic Learning-to-Rank. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '20). Association for Computing Machinery, New York, NY, USA, 429--438.

Digital Library

[40]

Geoff Norman. 2010. Likert scales, levels of measurement and the "laws" of statistics. Advances in Health Sciences Education, Vol. 15, 5 (2010), 625--632. https://doi.org/10.1007/s10459-010--9222-y

[41]

Alamir Novin and Eric Meyers. 2017. Making sense of conflicting science information: Exploring bias in the search engine result page. In CHIIR 2017 - Proceedings of the 2017 Conference Human Information Interaction and Retrieval. 175--184. https://doi.org/10.1145/3020165.3020185

Digital Library

[42]

Heather L. O'Brien, Paul Cairns, and Mark Hall. 2018. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human Computer Studies, Vol. 112, July 2017 (2018), 28--39. https://doi.org/10.1016/j.ijhcs.2018.01.004

[43]

Maeve O'Brien and Mark T. Keane. 2006. Modeling result-list searching in the world wide web: The role of relevance topologies and trust bias. In Proceedings of the 28th Annual Conference of the Cognitive Science Society. 1881--1886.

[44]

Bing Pan, Helene Hembrooke, Thorsten Joachims, Lori Lorigo, Geri Gay, and Laura Granka. 2007. In Google we trust: Users' decisions on rank, position, and relevance. Journal of Computer-Mediated Communication, Vol. 12, 3 (2007), 801--823. https://doi.org/10.1111/j.1083--6101.2007.00351.x

[45]

Gordon Pennycook and David G. Rand. 2019. Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, Vol. 188, June 2018 (2019), 39--50. https://doi.org/10.1016/j.cognition.2018.06.011

[46]

Richard E Petty and Pablo Brinol. 2010. Attitude Change. In Advanced social psychology: The state of the science, R. F. Baumeister and E. J. Finkel (Eds.). Oxford University Press, Chapter 7, 217--259. https://doi.org/10.1016/B978-0--12--375000--6.00040--9

[47]

Frances A. Pogacar, Amira Ghenai, Mark D. Smucker, and Charles L.A. Clarke. 2017. The Positive and Negative Influence of Search Results on People's Decisions about the Efficacy of Medical Treatments. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR '17). Association for Computing Machinery, New York, NY, USA, 209--216. https://doi.org/10.1145/3121050.3121074

Digital Library

[48]

ProCon/EncyclopaediaBritannica. 2021. ProCon. https://www.procon.org/

[49]

Prolific. [n.d.]. Prolific. https://www.prolific.co/

[50]

Emily Pronin, Daniel Y. Lin, and Lee Ross. 2002. The bias blind spot: Perceptions of bias in self versus others., 369--381 pages. https://doi.org/10.1177/0146167202286008

[51]

Cornelius Puschmann. 2018. Beyond the Bubble: Assessing the Diversity of Political Search Results. Digital Journalism, Vol. 7, 6 (2018), 1--20. https://doi.org/10.1080/21670811.2018.1539626

[52]

Qualtrics. [n.d.]. Qualtrics. https://www.qualtrics.com/

[53]

Piotr Sapiezynski, Wesley Zeng, Ronald E. Robertson, Alan Mislove, and Christo Wilson. 2019. Quantifying the Impact of User Attentionon Fair Group Representation in Ranked Lists. In Companion Proceedings of The 2019 World Wide Web Conference (WWW '19). Association for Computing Machinery, New York, NY, USA, 553--562. https://doi.org/10.1145/3308560.3317595

Digital Library

[54]

Keith E. Stanovich and Richard F. West. 1997. Reasoning Independently of Prior Belief and Individual Differences in Actively Open-Minded Thinking. Journal of Educational Psychology, Vol. 89, 2 (1997), 342--357. https://doi.org/10.1037/0022-0663.89.2.342

[55]

Zakary L. Tormala and Derek D. Rucker. 2018. Attitude certainty: Antecedents, consequences, and new directions. Consumer Psychology Review, Vol. 1, 1 (2018), 72--89. https://doi.org/10.1002/arcp.1004

[56]

Don van den Bergh, Johnny van Doorn, Maarten Marsman, Tim Draws, Erik Jan van Kesteren, Koen Derks, Fabian Dablander, Quentin F. Gronau, vS imon Kucharský, Akash R.Komarlu Narendra Gupta, Alexandra Sarafoglou, Jan G. Voelkel, Angelika Stefan, Alexander Ly, Max Hinne, Dora Matzke, and Eric Jan Wagenmakers. 2020. A Tutorial on Conducting and Interpreting a Bayesian ANOVA in JASP. Annee Psychologique, Vol. 120, 1 (2020), 73--96. https://doi.org/10.3917/anpsy1.201.0073

[57]

Eric Jan Wagenmakers, Maarten Marsman, Tahira Jamil, Alexander Ly, Josine Verhagen, Jonathon Love, Ravi Selker, Quentin F. Gronau, Martin vS mí ra, Sacha Epskamp, Dora Matzke, Jeffrey N. Rouder, and Richard D. Morey. 2018. Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications. Psychonomic Bulletin and Review, Vol. 25, 1 (2018), 35--57. https://doi.org/10.3758/s13423-017--1343--3

[58]

Ryen W. White. 2013. Beliefs and biases in web search. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '13). Association for Computing Machinery, New York, NY, USA, 3--12. https://doi.org/10.1145/2484028.2484053

Digital Library

[59]

Ryen W. White and Ahmed Hassan. 2014. Content bias in online health search. ACM Transactions on the Web, Vol. 8, 4 (2014), 1--33. https://doi.org/10.1145/2663355

Digital Library

[60]

Ryen W. White and Eric Horvitz. 2014. Belief dynamics in web search. Journal of the Association for Information Science and Technology, Vol. 65, 11 (2014), 2165--2178. https://doi.org/10.1002/asi.23128

Digital Library

[61]

Yongkai Wu, Lu Zhang, and Xintao Wu. 2018. On discrimination discovery and removal in ranked data using causal graph. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, New York, NY, USA, 2536--2544. https://doi.org/10.1145/3219819.3220087 arxiv: 1803.01901

Digital Library

[62]

Yusuke Yamamoto and Satoshi Shimada. 2016. Can disputed topic suggestion enhance user consideration of information credibility in web search?. In HT 2016 - Proceedings of the 27th ACM Conference on Hypertext and Social Media. Association for Computing Machinery, New York, NY, USA, 169--177. https://doi.org/10.1145/2914586.2914592

Digital Library

[63]

Ke Yang and Julia Stoyanovich. 2017. Measuring fairness in ranked outputs. In Proceedings of the 29th International Conference on Scientific and Statistical Database Management (SSDBM '17). Association for Computing Machinery, New York, NY, USA, 1--6. https://doi.org/10.1145/3085504.3085526

Digital Library

[64]

Ke Yang, Julia Stoyanovich, Abolfazl Asudeh, Bill Howe, H. V. Jagadish, and Gerome Miklau. 2018. A nutritional label for rankings. Proceedings of the ACM SIGMOD International Conference on Management of Data Section 2 (2018), 1773--1776. https://doi.org/10.1145/3183713.3193568 arxiv: 1804.07890

Digital Library

[65]

Elad Yom-Tov, Susan Dumais, and Qi Guo. 2014. Promoting Civil Discourse Through Search Engine Diversity. Social Science Computer Review, Vol. 32, 2 (2014), 145--154. https://doi.org/10.1177/0894439313506838

Digital Library

[66]

Robert B. Zajonc. 1968. Attitudinal Effects of Mere Exposure. Journal of Personality and Social Psychology, Vol. 9, 2 PART 2 (1968), 1--27. https://doi.org/10.1037/h0025848

[67]

Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, and Ricardo Baeza-Yates. 2017. FA*IR: A Fair Top-k Ranking Algorithm. In Admit One. Association for Computing Machinery, New York, NY, USA, 1--18. https://doi.org/10.2307/j.ctt1b7x5pt.3

Cited By

Pathiyan Cherumanal SScholer FTrippas JSpina D(2024)Towards Investigating Biases in Spoken Conversational SearchCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3690156(61-66)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3690156
Wang BLiu JOosterhuis HBast HXiong C(2024)Cognitively Biased Users Interacting with Algorithmically Biased Results in Whole-Session Search on Debated TopicsProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672520(227-237)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672520
Ji KPathiyan Cherumanal STrippas JHettiachchi DSalim FScholer FSpina D(2024)Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational SearchAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680245(1-10)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680245
Show More Cited By

Index Terms

This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
2. Information systems
  1. World Wide Web
    1. Web searching and information discovery

Recommendations

Understanding How Algorithmic and Cognitive Biases in Web Search Affect User Attitudes on Debated Topics
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Web search increasingly provides a platform for users to seek advice on important personal decisions but may be biased in several different ways. One result of such biases is the search engine manipulation effect (SEME): when a list of search results ...
Reducing Click and Skip Errors in Search Result Ranking
WSDM '16: Proceedings of the Ninth ACM International Conference on Web Search and Data Mining

Search engines provide result summaries to help users quickly identify whether or not it is worthwhile to click on a result and read in detail. However, users may visit non-relevant results and/or skip relevant ones. These actions are usually harmful to ...
User rankings of search engine results

In this study, we investigate the similarities and differences between rankings of search results by users and search engines. Sixty-seven students took part in a 3-week-long experiment, during which they were asked to identify and rank the top 10 ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2021

2998 pages

ISBN:9781450380379

DOI:10.1145/3404835

General Chairs:
Fernando Diaz
(Google)
,
Chirag Shah
University of Washington
,
Torsten Suel
New York University
,
Program Chairs:
Pablo Castells
Universidad Autónoma de Madrid, Amazon
,
Rosie Jones
Spotify
,
Tetsuya Sakai
Waseda University

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2021

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Top Consortia for Knowledge and Innovation (TKI?s) of the Dutch ministry of economic affairs

Conference

SIGIR '21

Sponsor:

SIGIR

SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 11 - 15, 2021

Virtual Event, Canada

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
1,570
Total Downloads

Downloads (Last 12 months)310
Downloads (Last 6 weeks)56

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pathiyan Cherumanal SScholer FTrippas JSpina D(2024)Towards Investigating Biases in Spoken Conversational SearchCompanion Proceedings of the 26th International Conference on Multimodal Interaction10.1145/3686215.3690156(61-66)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3686215.3690156
Wang BLiu JOosterhuis HBast HXiong C(2024)Cognitively Biased Users Interacting with Algorithmically Biased Results in Whole-Session Search on Debated TopicsProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672520(227-237)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672520
Ji KPathiyan Cherumanal STrippas JHettiachchi DSalim FScholer FSpina D(2024)Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational SearchAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680245(1-10)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680245
Rieger ABredius FTheune MPera M(2024)From Potential to Practice: Intellectual Humility During Search on Debated TopicsProceedings of the 2024 Conference on Human Information Interaction and Retrieval10.1145/3627508.3638306(130-141)Online publication date: 10-Mar-2024
https://dl.acm.org/doi/10.1145/3627508.3638306
Epstein RLothringer MZankich V(2024)How a Daily Regimen of Operant Conditioning Might Explain the Power of the Search Engine Manipulation Effect (SEME)Behavior and Social Issues10.1007/s42822-023-00155-033:1(82-106)Online publication date: 5-Jan-2024
https://doi.org/10.1007/s42822-023-00155-0
Banerjee ABanik PWörndl W(2023)A review on individual and multistakeholder fairness in tourism recommender systemsFrontiers in Big Data10.3389/fdata.2023.11686926Online publication date: 10-May-2023
https://doi.org/10.3389/fdata.2023.1168692
Bauer CCarterette BFerro NFuhr NBeel JBreuer TClarke CCrescenzi ADemartini GDi Nunzio GDietz LFaggioli GFerwerda BFröbe MHagen MHanbury AHauff CJannach DKando NKanoulas EKnijnenburg BKruschwitz ULi MMaistro MMichiels LPapenmeier APotthast MRosso PSaid ASchaer PSeifert CSpina DStein BTintarev NUrbano JWachsmuth HWillemsen MZobel J(2023)Report on the Dagstuhl Seminar on Frontiers of Information Access Experimentation for Research and EducationACM SIGIR Forum10.1145/3636341.363635157:1(1-28)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1145/3636341.3636351
Draws TNatesan Ramamurthy KBaldini IDhurandhar APadhi ITimmermans BTintarev N(2023)Explainable Cross-Topic Stance Detection for Search ResultsProceedings of the 2023 Conference on Human Information Interaction and Retrieval10.1145/3576840.3578296(221-235)Online publication date: 19-Mar-2023
https://dl.acm.org/doi/10.1145/3576840.3578296
Hagiwara RYamamoto T(2023)Web Page Evaluation and Opinion Formation on Controversial Search TopicsLeveraging Generative Intelligence in Digital Libraries: Towards Human-Machine Collaboration10.1007/978-981-99-8085-7_17(188-203)Online publication date: 4-Dec-2023
https://dl.acm.org/doi/10.1007/978-981-99-8085-7_17
Wu ZDraws TCau FBarile FRieger ATintarev N(2023)Explaining Search Result Stances to Opinionated PeopleExplainable Artificial Intelligence10.1007/978-3-031-44067-0_29(573-596)Online publication date: 21-Oct-2023
https://doi.org/10.1007/978-3-031-44067-0_29
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten