research-article

Conversational review-based explanations for recommender systems: Exploring users’ query behavior

Authors:

Diana C. Hernandez-Bocanegra,

Jürgen ZieglerAuthors Info & Claims

CUI '21: Proceedings of the 3rd Conference on Conversational User Interfaces

Article No.: 1, Pages 1 - 11

https://doi.org/10.1145/3469595.3469596

Published: 27 July 2021 Publication History

Abstract

Providing explanations based on user reviews in recommender systems (RS) can increase users’ perception of system transparency. While static explanations are dominant, interactive explanatory approaches have emerged in explainable artificial intelligence (XAI), so that users are more likely to examine system decisions and get more arguments supporting system assertions. However, little attention has been paid to conversational approaches for explanations targeting end users. In this paper we explore how to design a conversational interface to provide explanations in a review-based RS, and present the results of a Wizard of Oz (WoOz) study that provided insights into the type of questions users might ask in such a context, as well as their perception of a system simulating such a dialog. Consequently, we propose a dialog management policy and user intents for explainable review-based RS, taking as an example the hotels domain.

References

[1]

Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y. Lim, and Mohan Kankanhalli. 2018. Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI 18. 1–18.

Digital Library

[2]

Abdallah Arioua and Madalina Croitoru. 2015. Formalizing Explanatory Dialogues. Scalable Uncertainty Management(2015), 282–297.

[3]

Abdallah Arioua, Madalina Croitoru, Laura Papaleo, Nathalie Pernelle, and Swan Rocher. 2016. On the Explanation of SameAs Statements Using Argumentation. Scalable Uncertainty Management(2016), 51–66. https://doi.org/10.1007/978-3-319-45856-4_4

[4]

Roland Bader, Wolfgang Woerndl, Andreas Karitnig, and Gerhard Leitner. 2012. Designing an explanation interface for proactive recommendations in automotive scenarios. In Proceedings of the 19th International Conference on User Modeling, Adaptation, and Personalization (UMAP’11). 92–104.

Digital Library

[5]

Sabrina Barko-Sherif, David Elsweiler, and Morgan Harvey. 2020. Conversational Agents for Recipe Recommendation. In Proceedings of the 2020 Conference on Human Information Interaction and Retrieval. 73–82. https://doi.org/10.1145/3343413.3377967

Digital Library

[6]

Konstantin Bauman, Bing Liu, and Alexander Tuzhilin. 2017. Aspect Based Recommendations: Recommending Items with the Most Valuable Aspects Based on User Reviews. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 717–725.

Digital Library

[7]

Jamal Bentahar, Bernard Moulin, and Micheline Belanger. 2010. A taxonomy of argumentation models used for knowledge representation. Artificial Intelligence Review 33, 3 (2010), 211–259.

Digital Library

[8]

Johannes Bjerva, Nikita Bhutani, Behzad Golshan, Wang-Chiew Tan, and Isabelle Augenstein. 2020. SubjQA: A Dataset for Subjectivity and Review Comprehension. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing EMNLP. 5480–5494. https://doi.org/10.18653/v1/2020.emnlp-main.442

[9]

Marco De Boni and Suresh Manandhar. 2020. Implementing clarification dialogues in open domain question answering. Natural Language Engineering 11, 4 (2020), 343–361. https://doi.org/10.1017/S1351324905003682

Digital Library

[10]

Dimitrios Buhalis and Emily Siaw Yen Cheng. 2020. Exploring the Use of Chatbots in Hotels: Technology Providers Perspective. Information and Communication Technologies in Tourism (2020), 231–242. https://doi.org/10.1007/978-3-030-36737-4_19

[11]

Giuseppe Carenini and Johanna D. Moore. 2006. Generating and evaluating evaluative arguments. In Artif. Intell., Vol. 170. 925–952.

[12]

Li Chen and Pearl Pu. 2014. Critiquing-based recommenders: survey and emerging trends. 22, 1–2(2014), 3085–3094.

[13]

Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. 2016. Towards Conversational Recommender Systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining KDD 16. 815–824. https://doi.org/10.1145/2939672.2939746

Digital Library

[14]

Oana Cocarascu, Antonio Rago, and Francesca Toni. 2019. Extracting Dialogical Explanations for Review Aggregations with Argumentative Dialogical Agents. In Proc. of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019).

Digital Library

[15]

Nils Dahlback, Arne Jonsson, and Lars Ahrenberg. 1993. Wizard of Oz Studies: Why and How. In Proceedings of the 1st Int. Conference on Intelligent User Interface. 193–200.

Digital Library

[16]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. (2019).

[17]

Rebecca Eynon, Chris Davies, and Wayne Holmes. 2012. Wizard of Oz for Multimodal Interfaces Design: Deployment Considerations. In Proceedings of the 8th International Conference on Networked Learning. 66–73.

[18]

Miguel Grinberg. 2020. Socket.IO. (2020). https://github.com/miguelgrinberg/Flask-SocketIO

[19]

Ivan Habernal and Iryna Gurevych. 2017. Argumentation mining in user-generated web discourse. In Computational Linguistics 43, Vol. 1. 125–179.

[20]

Diana C. Hernandez-Bocanegra, Tim Donkers, and Jürgen Ziegler. 2020. Effects of Argumentative Explanation Types on the Perception of Review-Based Recommendations. In Adjunct Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization (UMAP ’20 Adjunct).

[21]

Diana C Hernandez-Bocanegra and Juergen Ziegler. 2020. Explaining Review-Based Recommendations: Effects of Profile Transparency, Presentation Style and User Characteristics. Journal of Interactive Media 19, 3 (2020), 81–200. https://doi.org/10.1515/icom-2020-0021

[22]

Diana C. Hernandez-Bocanegra and Jürgen Ziegler. 2021. Effects of interactivity and presentation on review-based explanations for recommendations. In arXiv:2105.11794. http://arxiv.org/abs/2105.11794

Digital Library

[23]

Denis J. Hilton. 1990. Conversational processes and causal explanation. 107, 1 (1990), 65–81.

[24]

Dietmar Jannach, Ahtsham Manzoor, Wanling Cai, and Li Chen. 2020. A Survey on Conversational Recommender Systems. 1–35. https://doi.org/abs/2004.00646

[25]

Nitin Jindal and Bing Liu. 2006. Identifying comparative sentences in text documents. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval - SIGIR 06. 244–251. https://doi.org/10.1145/1148170.1148215

Digital Library

[26]

John F. Kelley. 1984. An Iterative Design Methodology for User-Friendly Natural Language Information Applications. In Transactions on Office Information Systems, Vol. 2. 26–41.

[27]

Lisa Klein. 1998. Evaluating the Potential of InteractiveMedia through a New Lens: Search versus Experience Goods. In Journal of Business Research, Vol. 41. 195–203.

[28]

Bart P. Knijnenburg, Martijn C. Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell. 2012. Explaining the User Experience of Recommender Systems. In User Modeling and User-Adapted Interaction. 441–504.

[29]

Pigi Kouki, James Schaffer, Jay Pujara, John O’Donovan, and Lise Getoor. 2019. Personalized Explanations for Hybrid Recommender Systems. In Proceedings of 24th International Conference on Intelligent User Interfaces (IUI 19). ACM, 379–390.

Digital Library

[30]

Béatrice Lamche, Ugur Adigüzel, and Wolfgang Wörndl. 2012. Interactive explanations in mobile shopping recommender systems. In Proceedings of the 4th International Workshop on Personalization Approaches in Learning Environments (PALE’14), held in conjunction with the 22nd International Conference on User Modeling, Adaptation, and Personalization (UMAP’14). 92–104.

[31]

J. Richard Landis and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics 33, 1 (1977), 159–174. Klagenfurt, Germany: SSOAR.

[32]

Raymond Li, Samira Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards Deep Conversational Recommendations. In 32nd Conference on Neural Information Processing Systems, NeurIPS 2018. 9725–9735.

[33]

Q. Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: Informing Design Practices for Explainable AI User Experiences. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems 9042(2020), 1–15. https://doi.org/10.1145/3313831.3376590

Digital Library

[34]

Nathalie Rose Lim, Patrick Saint-Dizier, and Rachel Roxas. 2009. Some challenges in the design of comparative and evaluative question answering systems. In In Proceedings of the 2009 Workshop on Knowledge and Reasoning for Answering Questions - KRAQ 09. 15–18. https://doi.org/10.3115/1697288.1697292

Digital Library

[35]

Peter Lipton. 1990. Contrastive explanation. Royal Institute of Philosophy Supplement 27 (1990), 247–266.

[36]

Yuping Liu and L J Shrum. 2002. What Is Interactivity and Is It Always Such a Good Thing? Implications of Definition, Person, and Situation for the Influence of Interactivity on Advertising Effectiveness. Journal of Advertising 31, 4 (2002), 53–64.

[37]

Benedikt Loepp, Katja Herrmanny, and Juergen Ziegler. 2015. Blended Recommending: Integrating Interactive Information Filtering and Algorithmic Recommender Techniques. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI 15. 975–984.

Digital Library

[38]

Benedikt Loepp, Tim Hussein, and Juergen Ziegler. 2014. Choice-based preference elicitation for collaborative filtering recommender systems. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI 14. 3085–3094.

Digital Library

[39]

Prashan Madumal, Tim Miller, Liz Sonenberg, and Frank Vetere. 2019. A Grounded Interaction Protocol for Explainable Artificial Intelligence. In Proc. of the 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019. 1–9.

[40]

Bella Martin and Bruce Hanington. 2012. Universal Methods of Design. Rockport Publishers, Beverly, MA.

[41]

Philipp Mayring. 2014. Qualitative Content Analysis: Theoretical Foundation, Basic Procedures and Software Solution. (2014). Klagenfurt, Germany: SSOAR.

[42]

D. Harrison McKnight, Vivek Choudhury, and Charles Kacmar. 2002. Developing and Validating Trust Measures for e-Commerce: An Integrative Typology. In Information Systems Research, Vol. 13.

[43]

Tim Miller. 2018. Explanation in Artificial Intelligence: Insights from the Social Sciences. Artificial Intelligence(2018).

[44]

Amit Mishra and Sanjay Kumar Jain. 2015. An Approach for Sentiment analysis of Complex Comparative Opinion Why Type Questions Asked on Product Review Sites. Computational Linguistics and Intelligent Text Processing Springer LNCS 9042(2015), 257–271.

[45]

Christof Monz. 2003. Document Retrieval in the Context of Question Answering. In Proceedings of the 25th European conference on IR research. 571–579.

[46]

Robert J. Moore and Raphael Arar. 2018. Conversational UX Design: An Introduction. Studies in Conversational UX Design(2018), 1–16. https://doi.org/10.1007/978-3-319-95579-7_1 Springer International Publishing.

[47]

Emanuela Moreale and Maria Vargas-Vera. 2004. A Question-Answering System Using Argumentation. MICAI 2004: Advances in Artificial Intelligence (2004), 400–409. https://doi.org/10.1007/978-3-540-24694-7_41

[48]

Khalil Ibrahim Muhammad, Aonghus Lawlor, and Barry Smyth. 2016. A Live-User Study of Opinionated Explanations for Recommender Systems. In Intelligent User Interfaces (IUI 16), Vol. 2. 256–260.

[49]

Philip J. Nelson. 1981. Consumer Information and Advertising. In Economics of Information. 42–77.

[50]

Hans Dybkjaer Niels Ole Bernsen and Laila Dybkjaer. 1993. Wizard of Oz prototyping: How and when. In In CCI Working Papers in Cognitive Science and HCI.

[51]

Florian Pecune, Shruti Murali, Vivian Tsai, Yoichi Matsuyama, and Justine Cassell. 2019. A Model of Social Explanations for a Conversational Movie Recommendation System. In Proceedings of the 7th International Conference on Human-Agent Interaction. 135–143. https://doi.org/10.1145/3349537.3351899

Digital Library

[52]

Pearl Pu, Li Chen, and Rong Hu. 2011. A user-centric evaluation framework for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems - RecSys 11. 157–164.

Digital Library

[53]

Silvia Quarteroni and Suresh Manandhar. 2008. Designing an interactive open-domain question answering system. Natural Language Engineering 15, 1 (2008), 73–95. https://doi.org/10.1017/S1351324908004919

Digital Library

[54]

Antonio Rago, Oana Cocarascu, Christos Bechlivanidis, and Francesca Toni. 2020. Argumentation as a Framework for Interactive Explanations for Recommendations. In Proceedings of the Seventeenth International Conference on Principles of Knowledge Representation and Reasoning. 805–815.

[55]

Andrew I Schein, Alexandrin Popescul, Lyle H Ungar, and David M Pennock. 2002. Methods and Metrics for Cold-Start Recommendations. In Proceedings of SIGIR 2002. 253–260.

Digital Library

[56]

Kacper Sokol and Peter Flach. 2020. One Explanation Does Not Fit All: The Promise of Interactive Explanations for Machine Learning Transparency. 34, 2 (2020), 235–250.

[57]

Ronnie Taib and Natalie Ruiz. 2007. Wizard of Oz for Multimodal Interfaces Design: Deployment Considerations. In Human-Computer Interaction. Interaction Design and Usability. 232–241.

[58]

Nava Tintarev. 2007. Explanations of recommendations. Proceedings of the 2007 ACM conference on Recommender systems, RecSys 07 (2007), 203–206.

Digital Library

[59]

Henning Wachsmuth, Martin Trenkmann, Benno Stein, Gregor Engels, and Tsvetomira Palakarska. 2014. A review corpus for argumentation analysis. In 15th International Conference on Intelligent Text Processing and Computational Linguistics. 115–127.

Digital Library

[60]

Douglas Walton. 2000. The Place of Dialogue Theory in Logic, Computer Science and Communication Studies. 123 (2000), 327–346.

[61]

Douglas Walton. 2004. A new dialectical theory of explanation. 7, 1 (2004), 71–89.

[62]

Douglas Walton. 2011. A dialogue system specification for explanation. 182, 3 (2011), 349–374.

[63]

Douglas Walton and Erik C. W. Krabbe. 1995. Commitment in dialogue: Basic concepts of interpersonal reasoning. State University of New York Press, New York.

[64]

Nan Wang, Hongning Wang, Yiling Jia, and Yue Yin. 2018. Explainable Recommendation via Multi-Task Learning in Opinionated Text Data. In Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 18. 165–174.

Digital Library

[65]

Daniel S. Weld and Gagan Bansal. 2019. The challenge of crafting intelligible intelligence. 62, 6 (2019), 70–79.

Digital Library

[66]

Markus Zanker and Martin Schoberegger. 2014. An empirical study on the persuasiveness of fact-based explanations for recommender systems. In Joint Workshop on Interfaces and Human Decision Making in Recommender Systems. 33–36.

[67]

Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W. Bruce Croft. 2018. Towards Conversational Search and Recommendation: System Ask, User Respond. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. 177–186. https://doi.org/10.1145/3269206.3271776

Digital Library

[68]

Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma.2014. Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In Proceedings of the 37th international ACM SIGIR conference on Research and development in information retrieval. 83–92.

Cited By

Jin YChen LCai WZhao X(2024)CRS-Que: A User-centric Evaluation Framework for Conversational Recommender SystemsACM Transactions on Recommender Systems10.1145/36315342:1(1-34)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3631534
Zhang THe X(2024)Enhancing Explainability in Medical AI: Developing Human-Centered Participatory Design CardsHCI International 2024 – Late Breaking Papers10.1007/978-3-031-76827-9_10(164-194)Online publication date: 31-Dec-2024
https://doi.org/10.1007/978-3-031-76827-9_10
Bertrand AEagan JMaxwell W(2023)Questioning the ability of feature-based explanations to empower non-experts in robo-advised financial decision-makingProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594053(943-958)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594053
Show More Cited By

Conversational review-based explanations for recommender systems: Exploring users’ query behavior
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms

Recommendations

Explaining Recommendations through Conversations: Dialog Model and the Effects of Interface Type and Degree of Interactivity
Explaining system-generated recommendations based on user reviews can foster users’ understanding and assessment of the recommended items and the recommender system (RS) as a whole. While up to now explanations have mostly been static, shown in a single ...
LLM-generated Explanations for Recommender Systems
UMAP Adjunct '24: Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization

Users are often confronted with situations where they have to decide in favor or against an offered item, like a book, movie, or recipe. Those suggested items are commonly determined by a recommender system, which considers personal preferences to ...
Knowledgeable Explanations for Recommender Systems
WI-IAT '10: Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 01

Recommender Systems (RS) serve online customers in identifying those items from a variety of choices that best match their needs and preferences. In this context explanations summarize the reasons why a specific item is proposed and strongly increase ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CUI '21: Proceedings of the 3rd Conference on Conversational User Interfaces

July 2021

262 pages

ISBN:9781450389983

DOI:10.1145/3469595

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

German Research Foundation (DFG)

Conference

CUI '21

CUI '21: CUI 2021 - 3rd Conference on Conversational User Interfaces

July 27 - 29, 2021

Bilbao (online), Spain

Acceptance Rates

Overall Acceptance Rate 34 of 100 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
539
Total Downloads

Downloads (Last 12 months)106
Downloads (Last 6 weeks)7

Reflects downloads up to 22 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jin YChen LCai WZhao X(2024)CRS-Que: A User-centric Evaluation Framework for Conversational Recommender SystemsACM Transactions on Recommender Systems10.1145/36315342:1(1-34)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3631534
Zhang THe X(2024)Enhancing Explainability in Medical AI: Developing Human-Centered Participatory Design CardsHCI International 2024 – Late Breaking Papers10.1007/978-3-031-76827-9_10(164-194)Online publication date: 31-Dec-2024
https://doi.org/10.1007/978-3-031-76827-9_10
Bertrand AEagan JMaxwell W(2023)Questioning the ability of feature-based explanations to empower non-experts in robo-advised financial decision-makingProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency10.1145/3593013.3594053(943-958)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3593013.3594053
Hernandez-Bocanegra DZiegler J(2023)Explaining Recommendations through Conversations: Dialog Model and the Effects of Interface Type and Degree of InteractivityACM Transactions on Interactive Intelligent Systems10.1145/357954113:2(1-47)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1145/3579541
Bertrand AViard TBelloum REagan JMaxwell W(2023)On Selective, Mutable and Dialogic XAI: a Review of What Users Say about Different Types of Interactive ExplanationsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581314(1-21)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581314
Traubinger VHeil SGrigera JGarrido AGaedke M(2023)In Search of Dark Patterns in ChatbotsChatbot Research and Design10.1007/978-3-031-54975-5_7(117-132)Online publication date: 22-Nov-2023
https://dl.acm.org/doi/10.1007/978-3-031-54975-5_7
Malandri LMercorio FMezzanzanica MNobani N(2022)ConvXAI: a System for Multimodal Interaction with Any Black-box ExplainerCognitive Computation10.1007/s12559-022-10067-715:2(613-644)Online publication date: 7-Nov-2022
https://doi.org/10.1007/s12559-022-10067-7

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents