research-article

Guidelines for Human-AI Interaction

Authors:

Saleema Amershi,

Mihaela Vorvoreanu,

Penny Collisson,

Paul N. Bennett,

Ruth Kikin-Gil,

Eric HorvitzAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 3, Pages 1 - 13

https://doi.org/10.1145/3290605.3300233

Published: 02 May 2019 Publication History

Abstract

Advances in artificial intelligence (AI) frame opportunities and challenges for user interface design. Principles for human-AI interaction have been discussed in the human-computer interaction community for over two decades, but more study and innovation are needed in light of advances in AI and the growing uses of AI technologies in human-facing applications. We propose 18 generally applicable design guidelines for human-AI interaction. These guidelines are validated through multiple rounds of evaluation including a user study with 49 design practitioners who tested the guidelines against 20 popular AI-infused products. The results verify the relevance of the guidelines over a spectrum of interaction scenarios and reveal gaps in our knowledge, highlighting opportunities for further research. Based on the evaluations, we believe the set of design guidelines can serve as a resource to practitioners working on the design of applications and features that harness AI technologies, and to researchers interested in the further development of human-AI interaction design principles.

Supplementary Material

ZIP File (paper003.zip)

The auxiliary material consists of a single zip file containing a PDF of the paper's appendix. In this appendix, we illustrate each of our 18 human-AI interaction design guidelines with three example applications and three example violations provided by our user study participants when testing the guidelines against popular AI-infused products. For each example we indicate the product category the participant was testing, but obscure the specific product names.

Download
213.57 KB

References

[1]

Alexa. 2018. Top sites in the United States. Retrieved July, 2018 from https://www.alexa.com/topsites/countries/US

[2]

Kathy Baxter. 2017. How to Meet User Expectations for Artificial Intelligence. Medium. Retrieved September, 2018 from https://medium.com/salesforce-ux/how-to-meet-userexpectations-for-artificial-intelligence-a51d3c82af6

[3]

Victoria Bellotti and Keith Edwards. 2001. Intelligibility and Accountability: Human Considerations in Context-Aware Systems. Human-- Computer Interaction 16, 2--4 (2001), 193--212.

Digital Library

[4]

Andrea Bunt, Matthew Lount, and Catherine Lauzon. 2012. Are Explanations Always Important?: A Study of Deployed, Low-cost Intelligent Interactive Systems. In Proc. IUI '12. ACM, New York, NY, USA, 169-- 178.

Digital Library

[5]

Margaret Burnett, Simone Stumpf, Jamie Macbeth, Stephann Makri, Laura Beckwith, Irwin Kwan, Anicia Peters, and William Jernigan. 2016. GenderMag: A method for evaluating software's gender inclusiveness. Interacting with Computers 28, 6 (2016), 760--787.

[6]

comScore. 2018. Latest rankings. Retrieved July, 2018 from https: //www.comscore.com/Insights/Rankings

[7]

Maartje de Graaf, Somaya Ben Allouch, and Jan van Dijk. 2017. Why Do They Refuse to Use My Robot?: Reasons for Non-Use Derived from a Long-Term Home Study. In Proc. HRI '17. ACM, New York, NY, USA, 224--233.

Digital Library

[8]

Defy Media. 2015. Damn You Auto Correct! Retrieved September, 2018 from http://www.damnyouautocorrect.com/

[9]

T. Deuschel and T. Scully. 2016. On the Importance of Spatial Perception for the Design of Adaptive User Interfaces. In 2016 IEEE 10th International Conference on Self-Adaptive and Self-Organizing Systems (SASO). 70--79.

[10]

Anind Dey, Jennifer Mankoff, Gregory Abowd, and Scott Carter. 2002. Distributed Mediation of Ambiguous Context in Aware Environments. In Proc. UIST '02. ACM, New York, NY, USA, 121--130.

Digital Library

[11]

Leah Findlater and Joanna McGrenere. 2004. A Comparison of Static, Adaptive, and Adaptable Menus. In Proc. CHI '04. ACM, New York, NY, USA, 89--96.

Digital Library

[12]

Krzysztof Z. Gajos, Mary Czerwinski, Desney S. Tan, and Daniel S. Weld. 2006. Exploring the Design Space for Adaptive Graphical User Interfaces. In Proc. AVI '06. ACM, New York, NY, USA, 201--208.

Digital Library

[13]

Krzysztof Z Gajos, Katherine Everitt, Desney S Tan, Mary Czerwinski, and Daniel S Weld. 2008. Predictability and accuracy in adaptive user interfaces. In CHI. ACM, 1271--1274.

Digital Library

[14]

Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining Collaborative Filtering Recommendations. In Proc. CSCW '00. ACM, New York, NY, USA, 241--250.

Digital Library

[15]

Setia Hermawati and Glyn Lawson. 2016. Establishing usability heuristics for heuristics evaluation in a specifc domain: Is there a consensus? Applied Ergonomics 56 (2016), 34 -- 51.

[16]

Kristina Höök. 2000. Steps to take before intelligent user interfaces become real. Interacting with Computers 12, 4 (2000), 409--426.

[17]

Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. In Proc. CHI '99. ACM, New York, NY, USA, 159--166.

Digital Library

[18]

Eric Horvitz, Jack Breese, David Heckerman, David Hovel, and Koos Rommelse. 1998. The Lumière Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users. In Proc. UAI '98. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 256--265.

Digital Library

[19]

IDC Corporate USA. 2018. IDC: The premier global market intelligence firm. Retrieved September, 2018 from https://www.idc.com/

[20]

Anthony Jameson. 2008. Adaptive interfaces and agents. In The human-computer interaction handbook: Fundamentals, evolving technologies and emerging applications (2nd ed.), Andrew Sears and Julie A. Jacko (Eds.). CRC Press, Boca Raton, FL, 433--458.

Digital Library

[21]

Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of Explanatory Debugging to Personalize Interactive Machine Learning. In Proc. IUI '15. ACM, New York, NY, USA, 126--137.

Digital Library

[22]

Chia-Jung Lee, Jaime Teevan, and Sebastian de la Chica. 2014. Characterizing Multi-click Search Behavior and the Risks and Opportunities of Changing Results During Use. In Proc. SIGIR '14. ACM, New York, NY, USA, 515--524.

Digital Library

[23]

Brian Y. Lim and Anind K. Dey. 2009. Assessing Demand for Intelligibility in Context-aware Applications. In Proc. UbiComp '09. ACM, New York, NY, USA, 195--204.

Digital Library

[24]

Josh Lovejoy. 2018. The UX of AI. Google Design. Retrieved September, 2018 from https://design.google/library/ux-ai/

[25]

Josh Lovejoy and Jess Holbrook. 2017. Human-Centered Machine Learning. 7 steps to stay focused on the user when designing with ML. Medium. Retrieved September, 2018 from https://medium.com/google-design/human-centered-machinelearning-a770d10562cd

[26]

Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf Between User Expectation and Experience of Conversational Agents. In Proc. CHI '16. ACM, New York, NY, USA, 5286--5297.

Digital Library

[27]

Aaron Maitland and Stanley Presser. 2016. How Accurately Do Different Evaluation Methods Predict the Reliability of Survey Questions? Journal of Survey Statistics and Methodology 4, 3 (2016), 362--381.

[28]

Joseph A Maxwell. 2012. Qualitative research design: An interactive approach. Vol. 41. Sage publications.

[29]

Chelsea Myers, Anushay Furqan, Jessica Nebolsky, Karina Caro, and Jichen Zhu. 2018. Patterns for How Users Overcome Obstacles in Voice User Interfaces. In Proc. CHI '18. ACM, New York, NY, USA, Article 6, 7 pages.

Digital Library

[30]

Jakob Nielsen. 1992. Finding Usability Problems Through Heuristic Evaluation. In Proc. CHI '92. ACM, New York, NY, USA, 373--380.

Digital Library

[31]

Jakob Nielsen and Rolf Molich. 1990. Heuristic Evaluation of User Interfaces. In Proc. CHI '90. ACM, New York, NY, USA, 249--256.

Digital Library

[32]

D.A. Norman. 1988. The psychology of everyday things. Basic Books, New York.

[33]

Donald A. Norman. 1994. How Might People Interact with Agents. Commun. ACM 37, 7 (July 1994), 68--71.

Digital Library

[34]

Kristen Olson. 2010. An examination of questionnaire evaluation by expert reviewers. Field Methods 22, 4 (2010), 295--318.

[35]

Martin Porcheron, Joel E. Fischer, Stuart Reeves, and Sarah Sharples. 2018. Voice Interfaces in Everyday Life. In Proc. CHI '18. ACM, New York, NY, USA, Article 640, 12 pages.

Digital Library

[36]

Emilee Rader, Kelley Cotter, and Janghee Cho. 2018. Explanations As Mechanisms for Supporting Algorithmic Transparency. In Proc. CHI '18. ACM, New York, NY, USA, Article 103, 13 pages.

Digital Library

[37]

Ruth Ravichandran, Sang-Wha Sien, Shwetak N. Patel, Julie A. Kientz, and Laura R. Pina. 2017. Making Sense of Sleep Sensors: How Sleep Sensing Technologies Support and Undermine Sleep Health. In Proc. CHI '17. ACM, New York, NY, USA, 6864--6875.

Digital Library

[38]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proc. KDD '16. ACM, New York, NY, USA, 1135--1144.

Digital Library

[39]

Katharine Schwab. 2017. 10 Principles For Design In The Age Of AI. Retrieved September, 2018 from https://www.fastcompany.com/3067632/10-principles-for-designin-the-age-of-ai

[40]

SimilarWeb. 2018. Top websites ranking. Retrieved July, 2018 from https://www.similarweb.com/top-websites/united-states

[41]

Jack Stewart. 2018. Why Tesla's Autopilot Can't See a Stopped Firetruck. Retrieved September, 2018 from https: //www.wired.com/story/tesla-autopilot-why-crash-radar/

[42]

Erica Virtue. 2017. Designing with AI.

[43]

Jolita Vveinhardt and Evelina Gulbovaite. 2016. Expert evaluation of diagnostic instrument for personal and organizational value congruence. Journal of business ethics 136, 3 (2016), 481--501.

[44]

Daniel S Weld and Gagan Bansal. 2018. Intelligible Artificial Intelligence. arXiv preprint arXiv:1803.04263 (2018).

[45]

Rayoung Yang, Eunice Shin, Mark W. Newman, and Mark S. Ackerman. 2015. When Fitness Trackers Don'T 'Fit': End-user Difficulties in the Assessment of Personal Tracking Device Accuracy. In Proc. UbiComp '15. ACM, New York, NY, USA, 623--634.

Digital Library

Cited By

Zeng JChen KWang RLi YFan MWu KQi XWang L(2025)ContractMind: Trust-calibration interaction design for AI contract review toolsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103411196(103411)Online publication date: Feb-2025
https://doi.org/10.1016/j.ijhcs.2024.103411
Ding WLin XZarro MDing WLin XZarro M(2025)Interaction Design and Design PatternsInformation Architecture and UX Design10.1007/978-3-031-72138-0_7(99-120)Online publication date: 8-Jan-2025
https://doi.org/10.1007/978-3-031-72138-0_7
Lee HGao LYang SForlizzi JDas SBalzarotti DXu W(2024)"I don't know if we're doing good. I don't know if we're doing bad": investigating how practitioners scope, motivate, and conduct privacy work when developing AI productsProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699173(4873-4890)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699173
Show More Cited By

Index Terms

Guidelines for Human-AI Interaction
1. Computing methodologies
  1. Artificial intelligence
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Assessing Human-AI Interaction Early through Factorial Surveys: A Study on the Guidelines for Human-AI Interaction
This work contributes a research protocol for evaluating human-AI interaction in the context of specific AI products. The research protocol enables UX and HCI researchers to assess different human-AI interaction solutions and validate design decisions ...
Human-AI Interaction and AI Avatars
HCI International 2023 – Late Breaking Papers
Abstract
Human-Computer Interaction has been evolving rapidly with the advancement of artificial intelligence and metaverse. Human-AI Interaction is a new area in Human-Computer Interaction. In this paper, we look at AI avatars, which are human-like ...
Participatory Evaluation of Human-Data Interaction Design Guidelines
Human-Computer Interaction – INTERACT 2019
Abstract
The design of visual analytics tools for facilitating human-data interaction (HDI) plays a key role to help people identifying useful knowledge from large masses of data. Designing data visualization based on guidelines is relevant. However, it is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

934
Total Citations
View Citations
25,728
Total Downloads

Downloads (Last 12 months)6,388
Downloads (Last 6 weeks)655

Reflects downloads up to 07 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zeng JChen KWang RLi YFan MWu KQi XWang L(2025)ContractMind: Trust-calibration interaction design for AI contract review toolsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103411196(103411)Online publication date: Feb-2025
https://doi.org/10.1016/j.ijhcs.2024.103411
Ding WLin XZarro MDing WLin XZarro M(2025)Interaction Design and Design PatternsInformation Architecture and UX Design10.1007/978-3-031-72138-0_7(99-120)Online publication date: 8-Jan-2025
https://doi.org/10.1007/978-3-031-72138-0_7
Lee HGao LYang SForlizzi JDas SBalzarotti DXu W(2024)"I don't know if we're doing good. I don't know if we're doing bad": investigating how practitioners scope, motivate, and conduct privacy work when developing AI productsProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699173(4873-4890)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699173
Marusich LBakdash JZhou YKantarcioglu MSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Using AI uncertainty quantification to improve human decision-makingProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693491(34949-34960)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693491
Lindauer MKarl FKlier AMoosbauer JTornede AMueller AHutter FFeurer MBischl BSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)PositionProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693301(30566-30584)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693301
Wang XWu Y(2024)Balancing Innovation and Regulation in the Age of Generative Artificial IntelligenceJournal of Information Policy10.5325/jinfopoli.14.2024.001214Online publication date: 2-Jul-2024
https://doi.org/10.5325/jinfopoli.14.2024.0012
Sekarini SSelvabaskar S(2024)AI-Powered BrandingIntegrating AI-Driven Technologies Into Service Marketing10.4018/979-8-3693-7122-0.ch002(19-48)Online publication date: 26-Jul-2024
https://doi.org/10.4018/979-8-3693-7122-0.ch002
LaFrance J(2024)Navigating Educational Leadership Research With Generative Artificial IntelligenceReimagining the P-20 Landscape for School Leadership Learning10.4018/979-8-3693-6220-4.ch006(139-168)Online publication date: 27-Dec-2024
https://doi.org/10.4018/979-8-3693-6220-4.ch006
Sindiramutty SPrabagaran KAkbar RHussain MMalik N(2024)Generative AI for Secure User Interface (UI) DesignReshaping CyberSecurity With Generative AI Techniques10.4018/979-8-3693-5415-5.ch010(333-394)Online publication date: 13-Sep-2024
https://doi.org/10.4018/979-8-3693-5415-5.ch010
Eswaran UEswaran V(2024)AI-Driven Cross-Platform DesignNavigating Usability and User Experience in a Multi-Platform World10.4018/979-8-3693-2337-3.ch002(19-48)Online publication date: 15-Nov-2024
https://doi.org/10.4018/979-8-3693-2337-3.ch002
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents