research-article

Effects of Argumentative Explanation Types on the Perception of Review-Based Recommendations

Authors:

Diana C. Hernandez-Bocanegra,

Jürgen ZieglerAuthors Info & Claims

UMAP '20 Adjunct: Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization

Pages 219 - 225

https://doi.org/10.1145/3386392.3399302

Published: 13 July 2020 Publication History

Abstract

Recommender systems have achieved considerable maturity and accuracy in recent years. However, the rationale behind recommendations mostly remains opaque. Providing textual explanations based on user reviews may increase users' perception of transparency and, by that, overall system satisfaction. However, little is known about how these explanations can be effectively and efficiently presented to the user. In the following paper, we present an empirical study conducted in the domain of hotels to investigate the effect of different textual explanation types on, among others, perceived system transparency and trustworthiness, as well as the overall assessment of explanation quality. The explanations presented to participants follow an argument-based design, which we propose to provide a rationale to support a recommendation in a structured way. Our results show that people prefer explanations that include an aggregation using percentages of other users' opinions, over explanations that only include a brief summary of opinions. The results additionally indicate that user characteristics such as social awareness may influence the perception of explanation quality.

Supplementary Material

VTT File (3386392.3399302.vtt)

Download
12.94 KB

MP4 File (3386392.3399302.mp4)

Supplemental Video

Download
47.03 MB

References

[1]

Roland Bader, Wolfgang Woerndl, Andreas Karitnig, and Gerhard Leitner. 2012. Designing an explanation interface for proactive recommendations in automotive scenarios. In Proceedings of the 19th International Conference on User Modeling, Adaptation, and Personalization (UMAP'11). 92--104.

Digital Library

[2]

Jacob A. Burack, Tara Flanagan, Terry Peled, Hazel M. Sutton, Catherine Zygmuntowicz, and Jody T. Manly. 2006. Social Perspective-Taking Skills in Maltreated Children and Adolescents. Developmental Psychology, Vol. 42, 2 (2006), 207--217.

[3]

Giuseppe Carenini, Jackie Chi Kit Cheung, and Adam Pauls. 2013. Multi document summarization of evaluative text. In Computational Intelligence, Vol. 29. 545--574.

[4]

Giuseppe Carenini and Johanna D. Moore. 2006. Generating and evaluating evaluative arguments. In Artif. Intell., Vol. 170. 925--952.

[5]

Michael Chandler. 1973. Egocentrism and Antisocial Behavior: The Assessment and Training of Social Perspective-Taking Skills. Developmental Psychology, Vol. 9, 3 (1973), 326--332.

[6]

Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. 2018. Neural attentional rating regression with review-level explanations. In Proceedings of the 2018 World Wide Web Conference on World Wide Web. International World Wide Web Conferences Steering Committee. 1583--1592.

Digital Library

[7]

Hanxiong Chen, Xu Chen, Shaoyun Shi, and Yongfeng Zhang. 2019. Generate Natural Language Explanations for Recommendation. In Proceedings of SIGIR 2019 Workshop on Explainable Recommendation and Search (EARS19) .

[8]

Felipe Costa, Sixun Ouyang, Peter Dolog, and Aonghus Lawlor. 2018. Automatic Generation of Natural Language Explanations. In Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion. 57:1--57:2.

Digital Library

[9]

Michael J. Driver, Kenneth E. Brousseau, and Phil L. Hunsaker. 1990. The dynamic decision maker. (1990).

[10]

Collaborative for Academic Social and Emotional Learning. 2013. 2013 CASEL guide: Effective social and emotional learning programs - Preschool and elementary school edition. (2013).

[11]

Shima Gerani, Yashar Mehdad, Giuseppe Carenini, Raymond T. Ng, and Bita Nejat. 2014. Abstractive Summarization of Product Reviews Using Discourse Structure. In Empirical Methods in Natural Language Processing, Vol. 53. 1602--1613.

[12]

Anindya Ghose and Panagiotis G. Ipeirotis. 2011. Estimating the helpfulness and economic impact of product reviews: Mining text and reviewer characteristics. Journal of Revenue and Pricing Management, Vol. 23 (2011), 1498--1512.

[13]

Ivan Habernal and Iryna Gurevych. 2017. Argumentation mining in user-generated web discourse. In Computational Linguistics 43, Vol. 1. 125--179.

Digital Library

[14]

Katherine Hamilton, Shin-I Shih, and Susan Mohammed. 2016. The Development and Validation of the Rational and Intuitive Decision Styles Scale. Journal of Personality Assessment, Vol. 98, 5 (2016), 523--535.

[15]

Xiangnan He, Tao Chen, Min-Yen Kan, and Xiao Chen. 2015. Trirank: Review aware explainable recommendation by modeling aspects. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. ACM, 1661--1670.

Digital Library

[16]

Ya-Han Hu, Yen-Liang Chen, and Hui-Ling Chou. 2017. Opinion mining from online hotel reviews: A text summarization approach. In Information Processing and Management, Vol. 53. 436--449.

Digital Library

[17]

Bart P. Knijnenburg, Martijn C. Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell. 2012. Explaining the User Experience of Recommender Systems. In User Modeling and User-Adapted Interaction. 441--504.

[18]

Béatrice Lamche, Ugur Adigüzel, and Wolfgang Wörndl. 2012. Interactive explanations in mobile shopping recommender systems. In Proceedings of the 4th International Workshop on Personalization Approaches in Learning Environments (PALE'14), held in conjunction with the 22nd International Conference on User Modeling, Adaptation, and Personalization (UMAP'14). 92--104.

[19]

D. Harrison McKnight, Vivek Choudhury, and Charles Kacmar. 2002. Developing and Validating Trust Measures for e-Commerce: An Integrative Typology. In Information Systems Research, Vol. 13.

[20]

Susan M. Mudambi and David Schuff. 2010. What makes a helpful online review? A study of customer reviews on amazon.com. MIS Quarterly (2010), 185--200.

[21]

Khalil Ibrahim Muhammad, Aonghus Lawlor, and Barry Smyth. 2016. A Live-User Study of Opinionated Explanations for Recommender Systems. In Intelligent User Interfaces (IUI 16), Vol. 2. 256--260.

[22]

Rosemary Pacini and Seymour Epstein. 1999. The relation of rational and experiential information processing styles to personality, basic beliefs, and the ratio-bias phenomenon. Journal of Personality and Social Psychology, Vol. 76 (1999), 972--987.

[23]

Bo Pang and Lillian Lee. 2008. Opinion Mining and Sentiment Analysis. In Foundations and Trends in Information Retrieval, Vol. 2. 1--135.

Digital Library

[24]

Richard E. Petty and John T. Cacioppo. 1986. Communication and persuasion: Central and peripheral routes to attitude change .Springer-Verlag, New York.

[25]

Pearl Pu, Li Chen, and Rong Hu. 2011. A user-centric evaluation framework for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems - RecSys 11. 157--164.

Digital Library

[26]

Janet A. Sniezek and Timothy Buckley. 1995. Cueing and Cognitive Conflict in Judge Advisor Decision Making. Organizational Behavior and Human Decision Processes, Vol. 62, 2 (1995), 159--174.

[27]

Nava Tintarev and Judith Masthoff. 2012. Evaluating the effectiveness of explanations for recommender systems. User Modeling and User-Adapted Interaction, Vol. 22 (2012), 399--439.

Digital Library

[28]

Stephen E. Toulmin. 1958. The Uses of Argument. (1958).

[29]

Jesse Vig, Shilad Sen, and John Riedl. 2009. Tagsplanations: explaining recommendations using tags. In Proceedings of the 14th international conference on Intelligent User Interfaces. ACM, 47--56.

Digital Library

[30]

Duane T. Wegener, Richard E. Petty, Kevin L. Blankenship, and Brian Detweiler-Bedell. 2010. Elaboration and numerical anchoring: Implications of attitude theories for consumer judgment and decision making. Consumer Psychology, Vol. 20 (2010), 5--16.

[31]

Bo Xiao and Izak Benbasat. 2007. ECommerce product recommendation agents: use, characteristics, and impact. MIS Quarterly, Vol. 31, 1 (2007), 137--209.

Digital Library

[32]

Ilan Yaniv and Maxim Milyavsky. 2007. Using advice from multiple sources to revise and improve judgments. Organizational Behavior and Human Decision Processes, Vol. 103 (2007), 104--120.

[33]

Markus Zanker and Martin Schoberegger. 2014. An empirical study on the persuasiveness of fact-based explanations for recommender systems. In Joint Workshop on Interfaces and Human Decision Making in Recommender Systems. 33--36.

[34]

Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma. 2014. Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In Proceedings of the 37th international ACM SIGIR conference on Research and development in information retrieval. 83--92.

Digital Library

Cited By

de Campos LFernández-Luna JHuete J(2024)An explainable content-based approach for recommender systems: a case study in journal recommendation for paper submissionUser Modeling and User-Adapted Interaction10.1007/s11257-024-09400-634:4(1431-1465)Online publication date: 6-Jun-2024
https://doi.org/10.1007/s11257-024-09400-6
Hernandez-Bocanegra DZiegler J(2023)Explaining Recommendations through Conversations: Dialog Model and the Effects of Interface Type and Degree of InteractivityACM Transactions on Interactive Intelligent Systems10.1145/357954113:2(1-47)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1145/3579541
Lu HMa WWang YZhang MWang XLiu YChua TMa S(2023)User Perception of Recommendation Explanation: Are Your Explanations What Users Need?ACM Transactions on Information Systems10.1145/356548041:2(1-31)Online publication date: 25-Jan-2023
https://dl.acm.org/doi/10.1145/3565480
Show More Cited By

Index Terms

Effects of Argumentative Explanation Types on the Perception of Review-Based Recommendations
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems

Recommendations

Conversational review-based explanations for recommender systems: Exploring users’ query behavior
CUI '21: Proceedings of the 3rd Conference on Conversational User Interfaces

Providing explanations based on user reviews in recommender systems (RS) can increase users’ perception of system transparency. While static explanations are dominant, interactive explanatory approaches have emerged in explainable artificial ...
Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Trust in a Recommender System (RS) is crucial for its overall success. However, it remains underexplored whether users trust personal recommendation sources (i.e. other humans) more than impersonal sources (i.e. conventional RS), and, if they do, ...
On the Negative Perception of Cross-domain Recommendations and Explanations
SIGIR '24: Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

Recommender systems typically operate within a single domain, for example, recommending books based on users' reading habits. If such data is unavailable, it may be possible to make cross-domain recommendations and recommend books based on user ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UMAP '20 Adjunct: Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization

July 2020

395 pages

ISBN:9781450379502

DOI:10.1145/3386392

Editors:
Tsvi Kuflik
University of Haifa, Israel
,
Ilaria Torre
University of Genoa, Italy
,
Robin Burke
University of Colorado, Boulder, USA
,
Cristina Gena
University of Turin, Italy

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

German Research Foundation (DFG)

Conference

UMAP '20

Sponsor:

UMAP '20: 28th ACM Conference on User Modeling, Adaptation and Personalization

July 14 - 17, 2020

Genoa, Italy

Acceptance Rates

Overall Acceptance Rate 162 of 633 submissions, 26%

Upcoming Conference

UMAP '25

Sponsor:
sigchi
sigchi

33rd ACM Conference on User Modeling, Adaptation and Personalization

June 16 - 19, 2025

New York City , NY , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
246
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)2

Reflects downloads up to 31 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

de Campos LFernández-Luna JHuete J(2024)An explainable content-based approach for recommender systems: a case study in journal recommendation for paper submissionUser Modeling and User-Adapted Interaction10.1007/s11257-024-09400-634:4(1431-1465)Online publication date: 6-Jun-2024
https://doi.org/10.1007/s11257-024-09400-6
Hernandez-Bocanegra DZiegler J(2023)Explaining Recommendations through Conversations: Dialog Model and the Effects of Interface Type and Degree of InteractivityACM Transactions on Interactive Intelligent Systems10.1145/357954113:2(1-47)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1145/3579541
Lu HMa WWang YZhang MWang XLiu YChua TMa S(2023)User Perception of Recommendation Explanation: Are Your Explanations What Users Need?ACM Transactions on Information Systems10.1145/356548041:2(1-31)Online publication date: 25-Jan-2023
https://dl.acm.org/doi/10.1145/3565480
Yang YLin YChen ZLei YLiu XZhang YSun YWang X(2022)SNPERS: A Physical Exercise Recommendation System Integrating Statistical Principles and Natural Language ProcessingElectronics10.3390/electronics1201006112:1(61)Online publication date: 23-Dec-2022
https://doi.org/10.3390/electronics12010061
Penha GKrikon EMurdock VElsweiler DKruschwitz ULudwig B(2022)Pairwise Review-Based Explanations for Voice Product SearchProceedings of the 2022 Conference on Human Information Interaction and Retrieval10.1145/3498366.3505828(300-304)Online publication date: 14-Mar-2022
https://dl.acm.org/doi/10.1145/3498366.3505828
Penha GKrikon EMurdock VAvula S(2022)Helping Voice Shoppers Make Purchase DecisionsExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519828(1-8)Online publication date: 27-Apr-2022
https://dl.acm.org/doi/10.1145/3491101.3519828
Chazette LKlos VHerzog FSchneider K(2022)Requirements on Explanations: A Quality Framework for Explainability2022 IEEE 30th International Requirements Engineering Conference (RE)10.1109/RE54965.2022.00019(140-152)Online publication date: Aug-2022
https://doi.org/10.1109/RE54965.2022.00019
Hernandez-Bocanegra DZiegler J(2021)Explaining Review-Based Recommendations: Effects of Profile Transparency, Presentation Style and User Characteristicsi-com10.1515/icom-2020-002119:3(181-200)Online publication date: 15-Jan-2021
https://doi.org/10.1515/icom-2020-0021
Hernandez-Bocanegra DZiegler J(2021)Effects of Interactivity and Presentation on Review-Based Explanations for RecommendationsHuman-Computer Interaction – INTERACT 202110.1007/978-3-030-85616-8_35(597-618)Online publication date: 30-Aug-2021
https://dl.acm.org/doi/10.1007/978-3-030-85616-8_35
Kunkel JNgo TZiegler JKrämer N(2021)Identifying Group-Specific Mental Models of Recommender Systems: A Novel Quantitative ApproachHuman-Computer Interaction – INTERACT 202110.1007/978-3-030-85610-6_23(383-404)Online publication date: 30-Aug-2021
https://dl.acm.org/doi/10.1007/978-3-030-85610-6_23

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten