skip to main content
10.1145/3571884.3597128acmconferencesArticle/Chapter ViewAbstractPublication PagescuiConference Proceedingsconference-collections
research-article
Open access

Trustworthy Embodied Conversational Agents for Healthcare: A Design Exploration of Embodied Conversational Agents for the periconception period at Erasmus MC

Published: 19 July 2023 Publication History

Abstract

This paper explores the potential implications of embodied conversational agents (ECAs) in healthcare, focusing on the impact of appearance and conversation style on trustworthiness. We conducted a Research through Design investigation of ECAs for supporting women during the periconception period and in pregnancy. The paper presents the results of a Wizard of Oz study in which two alternative prototypes, a chatbot, and an ECA, were tested in a tertiary hospital by 25 participants. Reflecting on the results we suggest that limited patients’ trust in ECAs maybe be beneficial for achieving trustworthy use of these agents in the healthcare context.

References

[1]
Alexander M Aroyo, Jan De Bruyne, Orian Dheu, Eduard Fosch-Villaronga, Aleksei Gudkov, Holly Hoch, Steve Jones, Christoph Lutz, Henrik Sætra, Mads Solberg, 2021. Overtrusting robots: Setting a research agenda to mitigate overtrust in automation. Paladyn, Journal of Behavioral Robotics 12, 1 (2021), 423–436.
[2]
Niels Ole Bernsen, Hans Dybkjær, and Laila Dybkjær. 1994. Wizard of oz prototyping: How and when. Proc. CCI Working Papers Cognit. Sci./HCI, Roskilde, Denmark (1994).
[3]
Timothy Bickmore, Ha Trinh, Reza Asadi, and Stefan Olafsson. 2018. Safety first: conversational agents for health care. In Studies in conversational UX design. Springer, 33–57.
[4]
Timothy W Bickmore, Laura M Pfeifer, and Brian W Jack. 2009. Taking the time to care: empowering low health literacy hospital patients with virtual nurse agents. In Proceedings of the SIGCHI conference on human factors in computing systems. 1265–1274.
[5]
Michael Bonfert, Nima Zargham, Florian Saade, Robert Porzel, and Rainer Malaka. 2021. An Evaluation of Visual Embodiment for Voice Assistants on Smart Displays. In CUI 2021-3rd Conference on Conversational User Interfaces. 1–11.
[6]
Sheryl Brahnam and Antonella De Angeli. 2012. Gender affordances of conversational agents. Interacting with Computers 24, 3 (2012), 139–153.
[7]
Cynthia Breazeal, Cory D Kidd, Andrea Lockerd Thomaz, Guy Hoffman, and Matt Berlin. 2005. Effects of nonverbal communication on efficiency and robustness in human-robot teamwork. In 2005 IEEE/RSJ international conference on intelligent robots and systems. IEEE, 708–713.
[8]
Lorainne Tudor Car, Dhakshenya Ardhithy Dhinagaran, Bhone Myint Kyaw, Tobias Kowatsch, Shafiq Joty, Yin-Leng Theng, Rifat Atun, 2020. Conversational agents in health care: scoping review and conceptual analysis. Journal of medical Internet research 22, 8 (2020), e17158.
[9]
Luciano Cavalcante Siebert, Maria Luce Lupetti, Evgeni Aizenberg, Niek Beckers, Arkady Zgonnikov, Herman Veluwenkamp, David Abbink, Elisa Giaccardi, Geert-Jan Houben, Catholijn M Jonker, 2022. Meaningful human control: Actionable properties for AI system development. AI and Ethics (2022), 1–15.
[10]
Leigh Clark, Nadia Pantidi, Orla Cooney, Philip Doyle, Diego Garaialde, Justin Edwards, Brendan Spillane, Emer Gilmartin, Christine Murad, Cosmin Munteanu, 2019. What makes a good conversation? Challenges in designing truly conversational agents. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
[11]
Enrico Coiera, Baki Kocaballi, John Halamka, and Liliana Laranjo. 2018. The digital scribe. NPJ digital medicine 1, 1 (2018), 1–5.
[12]
E Coiera, JI Westbrook, and JC Wyatt. 2006. Section 1: Health and Clinical Mangement: The Safety and Quality of Decision Support Systems. Yearbook of medical informatics 15, 01 (2006), 20–25.
[13]
Benjamin R Cowan, Nadia Pantidi, David Coyle, Kellie Morrissey, Peter Clarke, Sara Al-Shehri, David Earley, and Natasha Bandeira. 2017. " What can i help you with?" infrequent users’ experiences of intelligent personal assistants. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. 1–12.
[14]
Andrew J Cowell and Kay M Stanney. 2003. Embodiment and interaction guidelines for designing credible, trustworthy embodied conversational agents. In International workshop on intelligent virtual agents. Springer, 301–309.
[15]
Samuel Rhys Cox and Wei Tsang Ooi. 2022. Does Chatbot Language Formality Affect Users’ Self-Disclosure?. In Proceedings of the 4th Conference on Conversational User Interfaces. 1–13.
[16]
Nigel Crisp and Lincoln Chen. 2014. Global supply of health professionals. New England Journal of Medicine 370, 10 (2014), 950–957.
[17]
Ewart J De Visser, Marieke MM Peeters, Malte F Jung, Spencer Kohn, Tyler H Shaw, Richard Pak, and Mark A Neerincx. 2020. Towards a theory of longitudinal trust calibration in human–robot teams. International journal of social robotics 12, 2 (2020), 459–478.
[18]
Ahmed Fadhil, Yunlong Wang, and Harald Reiterer. 2019. Assistive conversational agent for health coaching: a validation study. Methods of information in medicine 58, 01 (2019), 009–023.
[19]
Kate Goddard, Abdul Roudsari, and Jeremy C Wyatt. 2012. Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association 19, 1 (2012), 121–127.
[20]
Batoul Hojeij, Sam Schoenmakers, Sten Willemsen, Lenie van Rossem, Andras Dinnyes, Melek Rousian, and Regine PM Steegers-Theunissen. 2023. The Effect of an eHealth Coaching Program (Smarter Pregnancy) on Attitudes and Practices Toward Periconception Lifestyle Behaviors in Women Attempting Pregnancy: Prospective Study. Journal of Medical Internet Research 25 (2023), e39321.
[21]
Shivani Kapania, Oliver Siy, Gabe Clapper, Azhagu Meena SP, and Nithya Sambasivan. 2022. ” Because AI is 100% right and safe”: User Attitudes and Sources of AI Authority in India. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–18.
[22]
Rabia Khan and Antonella De Angeli. 2009. The attractiveness stereotype in the evaluation of embodied conversational agents. In IFIP Conference on Human-Computer Interaction. Springer, 85–97.
[23]
Rabia Fatima Khan and Alistair Sutcliffe. 2014. Attractive agents are more persuasive. International Journal of Human-Computer Interaction 30, 2 (2014), 142–150.
[24]
Ahmet Baki Kocaballi, Shlomo Berkovsky, Juan C Quiroz, Liliana Laranjo, Huong Ly Tong, Dana Rezazadegan, Agustina Briatore, and Enrico Coiera. 2019. The personalization of conversational agents in health care: systematic review. Journal of medical Internet research 21, 11 (2019), e15360.
[25]
Rafal Kocielnik, Raina Langevin, James S George, Shota Akenaga, Amelia Wang, Darwin P Jones, Alexander Argyle, Callan Fockele, Layla Anderson, Dennis T Hsieh, 2021. Can I Talk to You about Your Social Needs? Understanding Preference for Conversational User Interface in Health. In CUI 2021-3rd Conference on Conversational User Interfaces. 1–10.
[26]
Bing Cai Kok and Harold Soh. 2020. Trust in robots: Challenges and opportunities. Current Robotics Reports 1, 4 (2020), 297–309.
[27]
Ross Koppel, Joshua P Metlay, Abigail Cohen, Brian Abaluck, A Russell Localio, Stephen E Kimmel, and Brian L Strom. 2005. Role of computerized physician order entry systems in facilitating medication errors. Jama 293, 10 (2005), 1197–1203.
[28]
Tobias Kowatsch, Theresa Schachner, Samira Harperink, Filipe Barata, Ullrich Dittler, Grace Xiao, Catherine Stanger, Florian v Wangenheim, Elgar Fleisch, Helmut Oswald, 2021. Conversational agents as mediating social actors in chronic disease management involving health care professionals, patients, and family members: multisite single-arm feasibility study. Journal of medical Internet research 23, 2 (2021), e25060.
[29]
Raina Langevin, Ross J Lordon, Thi Avrahami, Benjamin R Cowan, Tad Hirsch, and Gary Hsieh. 2021. Heuristic evaluation of conversational agents. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.
[30]
Liliana Laranjo, Adam G Dunn, Huong Ly Tong, Ahmet Baki Kocaballi, Jessica Chen, Rabia Bashir, Didi Surian, Blanca Gallego, Farah Magrabi, Annie YS Lau, 2018. Conversational agents in healthcare: a systematic review. Journal of the American Medical Informatics Association 25, 9 (2018), 1248–1258.
[31]
Jingyi Li, Michelle X Zhou, Huahai Yang, and Gloria Mark. 2017. Confiding in and listening to virtual agents: The effect of personality. In Proceedings of the 22nd International Conference on Intelligent User Interfaces. 275–286.
[32]
Sally Lindsay. 2019. Five approaches to qualitative comparison groups in health research: a scoping review. Qualitative health research 29, 3 (2019), 455–468.
[33]
Christine Lisetti, Ugan Yasavur, Claudia De Leon, Reza Amini, Ubbo Visser, and Naphtali Rishe. 2012. Building an on-demand avatar-based health intervention for behavior change. In Twenty-Fifth International FLAIRS Conference.
[34]
David D Luxton. 2020. Ethical implications of conversational agents in global public health. Bulletin of the World Health Organization 98, 4 (2020), 285.
[35]
Raju Maharjan, Darius Adam Rohani, Per Bækgaard, Jakob Bardram, and Kevin Doherty. 2021. Can We Talk? Design Implications for the Questionnaire-Driven Self-Report of Health and Wellbeing via Conversational Agent. In Proceedings of the 3rd Conference on Conversational User Interfaces (Bilbao (online), Spain) (CUI ’21). Association for Computing Machinery, New York, NY, USA, Article 5, 11 pages. https://doi.org/10.1145/3469595.3469600
[36]
Roger C Mayer, James H Davis, and F David Schoorman. 1995. An integrative model of organizational trust. Academy of management review 20, 3 (1995), 709–734.
[37]
Jean-Arthur Micoulaud-Franchi, Patricia Sagaspe, Etienne de Sevin, Stéphanie Bioulac, Alain Sauteraud, and Pierre Philip. 2016. Acceptability of embodied conversational agent in a health care context. In International conference on intelligent virtual agents. Springer, 416–419.
[38]
Madison Milne-Ives, Caroline de Cock, Ernest Lim, Melissa Harper Shehadeh, Nick de Pennington, Guy Mole, Eduardo Normando, Edward Meinert, 2020. The effectiveness of artificial intelligence conversational agents in health care: systematic review. Journal of medical Internet research 22, 10 (2020), e20346.
[39]
Adam S Miner, Arnold Milstein, Stephen Schueller, Roshini Hegde, Christina Mangurian, and Eleni Linos. 2016. Smartphone-based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA internal medicine 176, 5 (2016), 619–625.
[40]
David C Mohr, Justin K Benzer, and Gary J Young. 2013. Provider workload and quality of care in primary care settings: moderating role of relational climate. Medical care (2013), 108–114.
[41]
Onora O’Neill. 2018. Linking trust to trustworthiness. International Journal of Philosophical Studies 26, 2 (2018), 293–300.
[42]
Alisha Pradhan and Amanda Lazar. 2021. Hey Google, Do You Have a Personality? Designing Personality and Personas for Conversational Agents. In CUI 2021-3rd Conference on Conversational User Interfaces. 1–4.
[43]
Minjin Rheu, Ji Youn Shin, Wei Peng, and Jina Huh-Yoo. 2021. Systematic review: Trust-building factors and implications for conversational agent design. International Journal of Human–Computer Interaction 37, 1 (2021), 81–96.
[44]
Laurel D Riek. 2012. Wizard of oz studies in hri: a systematic review and new reporting guidelines. Journal of Human-Robot Interaction 1, 1 (2012), 119–136.
[45]
Paul Robinette, Wenchen Li, Robert Allen, Ayanna M Howard, and Alan R Wagner. 2016. Overtrust of robots in emergency evacuation scenarios. In 2016 11th ACM/IEEE international conference on human-robot interaction (HRI). IEEE, 101–108.
[46]
Melek Rousian, Sam Schoenmakers, Alex J Eggink, Dionne V Gootjes, Anton HJ Koning, Maria PH Koster, Annemarie GMGJ Mulders, Esther B Baart, Irwin KM Reiss, Joop SE Laven, 2021. Cohort profile update: the Rotterdam Periconceptional Cohort and embryonic and fetal measurements using 3D ultrasound and virtual reality techniques. International Journal of Epidemiology 50, 5 (2021), 1426–1427l.
[47]
Elayne Ruane, Abeba Birhane, and Anthony Ventresque. 2019. Conversational AI: Social and Ethical Considerations. In AICS. 104–115.
[48]
Rahime Belen Sağlam and Jason RC Nurse. 2020. Is your chatbot GDPR compliant? Open issues in agent design. In Proceedings of the 2nd Conference on Conversational User Interfaces. 1–3.
[49]
Maha Salem, Gabriella Lakatos, Farshid Amirabdollahian, and Kerstin Dautenhahn. 2015. Would you trust a (faulty) robot? Effects of error, task type and personality on human-robot cooperation and trust. In Proceedings of the tenth annual ACM/IEEE international conference on human-robot interaction. 141–148.
[50]
Jan Smeddinck, Kamila Wajda, Adeel Naveed, Leen Touma, Yuting Chen, Muhammad Abu Hasan, Muhammad Waqas Latif, and Robert Porzel. 2010. QuickWoZ: a multi-purpose wizard-of-oz framework for experiments with embodied conversational agents. In Proceedings of the 15th international conference on Intelligent user interfaces. 427–428.
[51]
Brendan Spillane, Emer Gilmartin, Christian Saam, and Vincent Wade. 2019. Issues relating to trust in care agents for the elderly. In Proceedings of the 1st International Conference on Conversational User Interfaces. 1–3.
[52]
Pieter Jan Stappers and Elisa Giaccardi. 2017. Research through design. In The encyclopedia of human-computer interaction. The Interaction Design Foundation, 1–94.
[53]
Vera Stara, Benjamin Vera, Daniel Bolliger, Lorena Rossi, Elisa Felici, Mirko Di Rosa, Michiel de Jong, Susy Paolini, 2021. Usability and acceptance of the embodied conversational agent Anne by people with dementia and their caregivers: exploratory study in home environment settings. JMIR mHealth and uHealth 9, 6 (2021), e25891.
[54]
Regine PM Steegers-Theunissen, John Twigt, Valerie Pestinger, and Kevin D Sinclair. 2013. The periconceptional period, reproduction and long-term health of offspring: the importance of one-carbon metabolism. Human reproduction update 19, 6 (2013), 640–655.
[55]
Yolande Strengers and Jenny Kennedy. 2021. The smart wife: Why Siri, Alexa, and other smart home devices need a feminist reboot.
[56]
Silke ter Stal, Lean Leonie Kramer, Monique Tabak, Harm op den Akker, and Hermie Hermens. 2020. Design features of embodied conversational agents in eHealth: a literature review. International Journal of Human-Computer Studies 138 (2020), 102409.
[57]
Silke ter Stal, Monique Tabak, Harm op den Akker, Tessa Beinema, and Hermie Hermens. 2020. Who do you prefer? The effect of age, gender and role on users’ first impressions of embodied conversational agents in eHealth. International Journal of Human–Computer Interaction 36, 9 (2020), 881–892.
[58]
Markus Thaler, Stephan Schlögl, and Aleksander Groth. 2020. Agent vs. Avatar: Comparing embodied conversational agents concerning characteristics of the uncanny valley. In 2020 IEEE International Conference on Human-Machine Systems (ICHMS). IEEE, 1–6.
[59]
Richard Tomsett, Alun Preece, Dave Braines, Federico Cerutti, Supriyo Chakraborty, Mani Srivastava, Gavin Pearson, and Lance Kaplan. 2020. Rapid trust calibration through interpretable and uncertainty-aware AI. Patterns 1, 4 (2020), 100049.
[60]
Aditya Nrusimha Vaidyam, Hannah Wisniewski, John David Halamka, Matcheri S Kashavan, and John Blake Torous. 2019. Chatbots and conversational agents in mental health: a review of the psychiatric landscape. The Canadian Journal of Psychiatry 64, 7 (2019), 456–464.
[61]
Pieter Van den Hombergh, Beat Künzi, Glyn Elwyn, Jan van Doremalen, Reinier Akkermans, Richard Grol, and Michel Wensing. 2009. High workload and job stress are associated with lower practice performance in general practice: an observational study in 239 general practices in the Netherlands. BMC Health Services Research 9, 1 (2009), 1–8.
[62]
Alastair van Heerden, Xolani Ntinga, and Khanya Vilakazi. 2017. The potential of conversational agents to provide a rapid HIV counseling and testing services. In 2017 international conference on the frontiers and advances in data science (FADS). IEEE, 80–85.
[63]
Anke Versluis, Kyma Schnoor, Niels H Chavannes, Esther PWA Talboom-Kamp, 2022. Direct Access for Patients to Diagnostic Testing and Results Using eHealth: Systematic Review on eHealth and Diagnostics. Journal of Medical Internet Research 24, 1 (2022), e29303.
[64]
Sarah Theres Völkel, Samantha Meindl, and Heinrich Hussmann. 2021. Manipulating and evaluating levels of personality perceptions of voice assistants through enactment-based dialogue design. In CUI 2021-3rd Conference on Conversational User Interfaces. 1–12.
[65]
Catharine Wang, Timothy Bickmore, Deborah J Bowen, Tricia Norkunas, MaryAnn Campion, Howard Cabral, Michael Winter, and Michael Paasche-Orlow. 2015. Acceptability and feasibility of a virtual counselor (VICKY) to collect family health histories. Genetics in Medicine 17, 10 (2015), 822–830.
[66]
Carolin Wienrich, Clemens Reitelbach, and Astrid Carolus. 2021. The trustworthiness of voice assistants in the context of healthcare investigating the effect of perceived expertise on the trustworthiness of voice assistants, providers, data receivers, and automatic speech recognition. Frontiers in Computer Science 3 (2021), 685250.
[67]
Runting Zhong and Mengyao Ma. 2022. Effects of communication style, anthropomorphic setting and individual differences on older adults using voice assistants in a health context. BMC geriatrics 22, 1 (2022), 1–13.

Cited By

View all

Index Terms

  1. Trustworthy Embodied Conversational Agents for Healthcare: A Design Exploration of Embodied Conversational Agents for the periconception period at Erasmus MC

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CUI '23: Proceedings of the 5th International Conference on Conversational User Interfaces
      July 2023
      504 pages
      ISBN:9798400700149
      DOI:10.1145/3571884
      This work is licensed under a Creative Commons Attribution International 4.0 License.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 July 2023

      Check for updates

      Author Tags

      1. conversational style
      2. embodied conversational agents
      3. healthcare
      4. research through design
      5. trustworthiness

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      CUI '23
      Sponsor:
      CUI '23: ACM conference on Conversational User Interfaces
      July 19 - 21, 2023
      Eindhoven, Netherlands

      Acceptance Rates

      Overall Acceptance Rate 34 of 100 submissions, 34%

      Upcoming Conference

      CUI '25
      ACM Conversational User Interfaces 2025
      July 7 - 9, 2025
      Waterloo , ON , Canada

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)817
      • Downloads (Last 6 weeks)93
      Reflects downloads up to 03 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media