research-article

Interface Design Optimization as a Multi-Armed Bandit Problem

Authors:

J. Derek Lomas,

Nikhil Poonwala,

Sharan Shodhan,

Emma BrunskillAuthors Info & Claims

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Pages 4142 - 4153

https://doi.org/10.1145/2858036.2858425

Published: 07 May 2016 Publication History

Abstract

"Multi-armed bandits" offer a new paradigm for the AI-assisted design of user interfaces. To help designers understand the potential, we present the results of two experimental comparisons between bandit algorithms and random assignment. Our studies are intended to show designers how bandits algorithms are able to rapidly explore an experimental design space and automatically select the optimal design configuration. Our present focus is on the optimization of a game design space. The results of our experiments show that bandits can make data-driven design more efficient and accessible to interface designers, but that human participation is essential to ensure that AI systems optimize for the right metric. Based on our results, we introduce several design lessons that help keep human design judgment in the loop. We also consider the future of human-technology teamwork in AI-assisted design and scientific inquiry. Finally, as bandits deploy fewer low-performing conditions than typical experiments, we discuss ethical implications for bandits in large-scale experiments in education.

Supplementary Material

MP4 File (p4142-lomas.mp4)

Download
278.67 MB

References

[1]

Agarwal, D., Chen, B.C., and Elango, P. (2009) Explore/Exploit Schemes for Web Content Optimization. Ninth IEEE International Conference on Data Mining, 1-10.

Digital Library

[2]

Auer, P., Cesa-Bianchi, N., and Fischer, P. (2002) Finite-time Analysis of the Multiarmed Bandit Problem. Machine Learning, 235-256.

Digital Library

[3]

Berry, D. (2011) Adaptive Clinical Trials: The Promise and the Caution. Journal of clinical oncology: official journal of the American Society of Clinical Oncology 29, 6. 603-6.

[4]

Birkett, A. (2015) When to Run Bandit Tests Instead of A/B/n Tests. http://conversionxl.com/bandit-tests/

[5]

Bostrom, N. (2003). Ethical issues in advanced artificial intelligence. Science Fiction and Philosophy: From Time Travel to Superintelligence, 277--284.

[6]

Brezzi, M. and Lai, T.L. (2002) Optimal learning and experimentation in bandit problems. Journal of Economic Dynamics and Control 27, 1. 87-108.

[7]

Card, S., Mackinlay, J., & Robertson, G. (1990). The design space of input devices. ACM CHI

Digital Library

[8]

Chapelle, O., & Li, L. (2011). An empirical evaluation of thompson sampling. InAdvances in neural information processing systems (pp. 2249--2257).

[9]

Crook, T., Frasca, B., Kohavi, R., & Longbotham, R. (2009, June). Seven pitfalls to avoid when running controlled experiments on the web. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 1105--1114). ACM.

Digital Library

[10]

Drachen, A. and Canossa, A. (2009) Towards Gameplay Analysis via Gameplay Metrics. ACM MindTrek, 202- 209.

Digital Library

[11]

Fogarty, J., Forlizzi, J., and Hudson, S.E. (2001) Aesthetic Information Collages: Generating Decorative Displays that Contain Information. ACM CHI

Digital Library

[12]

Gajos, K., & Weld, D. S. (2005). Preference elicitation for interface optimization. ACM UIST (pp. 173--182).

Digital Library

[13]

Gajos, K., Weld, D., and Wobbrock, J. DecisionTheoretic User Interface Generation. AAAI, (2008), 1532-1536.

Digital Library

[14]

Gittins, J. (1979) Bandit Processes and Dynamic Allocation Indicies. Journal of the Royal Statistical Society. Series B., 148-177.

[15]

Glaser, R. (1976). Components of a psychology of instruction: Toward a science of design. Review of Educational Research, 46(1), 1-24.

[16]

Hacker, S. (2014) Duolingo: Learning a Language While Translating the Web. PhD Thesis, Carnegie Mellon University School of Computer Science. May

[17]

Hauser, J.R., Urban, G.L., Liberali, G., and Braun, M. (2009) Website Morphing. Marketing Science. 28, 2, 202-223.

Digital Library

[18]

Khajah, M., Roads, B. D., Lindsey, R. V, Liu, Y., & Mozer, M. C. (2016). Designing Engaging Games Using Bayesian Optimization, ACM CHI.

Digital Library

[19]

Koedinger, K. R., Booth, J. L., Klahr, D. (2013) Instructional Complexity and the Science to Constrain It Science. 22 November 2013: Vol. 342 no. 6161 pp. 935-

[20]

Koedinger, K. R., Baker, R. S., Cunningham, K., Skogsholm, A., Leber, B., & Stamper, J. (2010). A data repository for the EDM community: The PSLC DataShop. Handbook of educational data mining, 43.

[21]

Kohavi, R., Longbotham, R., Sommerfield, D., and Henne, R.M. (2008) Controlled experiments on the web: survey and practical guide. Data Mining and Knowledge Discovery 18, 1 140-181.

Digital Library

[22]

Kohavi, R., Deng, A., Frasca, B., Longbotham, R., Walker, T., and Xu, Y. (2012) Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained. KDD

Digital Library

[23]

Kramer, Adam DI, Jamie E. Guillory, and Jeffrey T. Hancock. (2014) Experimental evidence of massivescale emotional contagion through social networks. PNAS

[24]

Lai, T. (1987) Adaptive treatment allocation and the multi-armed bandit problem. The Annals of Statistics; 15(3):1091-1114.

[25]

Lai, T., & Robbins, H. (1985). Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6, 4-22.

Digital Library

[26]

Li, L., Chu, W., Langford, J., & Schapire, R.E. (2010) A Contextual-Bandit Approach to Personalized News Article Recommendation. WWW

Digital Library

[27]

Liu, Y., Mandel, T., Brunskill, E., & Popovic, Z. (2014) Trading Off Scientific Knowledge and User Learning with Multi-Armed Bandits. Educational Data Mining

[28]

Liu, Y., Mandel, T., Brunskill, E., & Popovi, Z. (2014) Towards Automatic Experimentation of Educational Knowledge. ACM CHI

Digital Library

[29]

Lomas, D., Patel, K., Forlizzi, J. L., & Koedinger, K. R. (2013) Optimizing challenge in an educational game using large-scale design experiments. ACM CHI

Digital Library

[30]

Lomas, D. Harpstead, E., (2012) Design Space Sampling for the Optimization of Educational Games. Game User Experience Workshop, ACM CHI

[31]

Lomas, D. (2014). Optimizing motivation and learning with large-scale game design experiments (Unpublished Doctoral Dissertation). HCI Institute, Carnegie Mellon University.

[32]

Lomas, D., (2013). Digital Games for Improving Number Sense Retrieved from https://pslcdatashop. web.cmu.edu/Files?datasetId=445

[33]

Maclean, A., Young, R. M., Victoria, M. E., & Moran, T. P. (1991). Questions, Options, and Criteria: Elements of Design Space Analysis. Human Computer Interaction, 6, 201-250.

Digital Library

[34]

Manzi, J. (2012). Uncontrolled: The surprising payoff of trial-and-error for business, politics, and society. Basic Books.

[35]

Norman, D. (in preparation) Technology or People: Putting People Back in Charge. Jnd.org

[36]

Scott, S. (2010) A modern Bayesian look at the multiarmed bandit. Applied Stochastic Models in Business and Industry, 639-658.

Digital Library

[37]

Scott, S. (2014) Google Content Experiments https://support.google.com/analytics/answer/2844870?hl=en&ref_topic=2844866

[38]

Scott, S. L. (2015). Multi-armed bandit experiments in the online service economy. Applied Stochastic Models in Business and Industry, 31(1), 37--45.

Digital Library

[39]

Simon, H. (1969). The Sciences of the Artificial. Cambridge, MA.

Digital Library

[40]

Stamper, J., Lomas, D., Ching, D., Ritter, S., Koedinger, K., & Steinhart, J. (2012) The rise of the super experiment. EDM p. 196-200

[41]

Stampfer, E., Long, Y., Aleven, V., & Koedinger, K. R. (2011, January). Eliciting intelligent novice behaviors with grounded feedback in a fraction addition tutor. In Artificial Intelligence in Education (pp. 560--562). Springer Berlin Heidelberg.

Digital Library

[42]

Vermorel, J. & Mohri, M. (2005) Multi-armed bandit algorithms and empirical evaluation. Machine Learning: ECML 2005, 437-448.

Digital Library

[43]

Yannakakis, G. N., & Hallam, J. (2007). Towards optimizing entertainment in computer games. Applied Artificial Intelligence, 21(10), 933--971.

Digital Library

Cited By

Langerak TChristen SAlbaba MGebhardt CHolz CHilliges O(2024)MARLUI: Multi-Agent Reinforcement Learning for Adaptive Point-and-Click UIsProceedings of the ACM on Human-Computer Interaction10.1145/36611478:EICS(1-27)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3661147
Gaspar-Figueiredo DFernández-Diego MNuredini RAbrahao SInsfran E(2024)Reinforcement Learning-Based Framework for the Intelligent Adaptation of User InterfacesCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661329(40-48)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3661329
Mo GDudley JChan LLiao YOulasvirta AKristensson P(2024)Cooperative Multi-Objective Bayesian Design OptimizationACM Transactions on Interactive Intelligent Systems10.1145/365764314:2(1-28)Online publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1145/3657643
Show More Cited By

Index Terms

Interface Design Optimization as a Multi-Armed Bandit Problem
1. Human-centered computing

Recommendations

Adapting User Interfaces with Model-based Reinforcement Learning
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Adapting an interface requires taking into account both the positive and negative effects that changes may have on the user. A carelessly picked adaptation may impose high costs to the user – for example, due to surprise or relearning effort – or “trap”...
Guidelines for Human-AI Interaction
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Advances in artificial intelligence (AI) frame opportunities and challenges for user interface design. Principles for human-AI interaction have been discussed in the human-computer interaction community for over two decades, but more study and ...
A contextual-bandit approach to personalized news article recommendation
WWW '10: Proceedings of the 19th international conference on World wide web

Personalized web services strive to adapt their services (advertisements, news articles, etc.) to individual users by making use of both content and user information. Despite a few recent advances, this problem remains challenging for at least two ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

May 2016

6108 pages

ISBN:9781450333627

DOI:10.1145/2858036

General Chairs:
Jofish Kaye
Yahoo
,
Allison Druin
University of Maryland / National Park Service
,
Program Chairs:
Cliff Lampe
University of Michigan
,
Dan Morris
Microsoft
,
Juan Pablo Hourcade
University of Iowa

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

US Department of Education

Conference

CHI'16

Sponsor:

SIGCHI

CHI'16: CHI Conference on Human Factors in Computing Systems

May 7 - 12, 2016

California, San Jose, USA

Acceptance Rates

CHI '16 Paper Acceptance Rate 565 of 2,435 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
980
Total Downloads

Downloads (Last 12 months)93
Downloads (Last 6 weeks)4

Reflects downloads up to 23 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Langerak TChristen SAlbaba MGebhardt CHolz CHilliges O(2024)MARLUI: Multi-Agent Reinforcement Learning for Adaptive Point-and-Click UIsProceedings of the ACM on Human-Computer Interaction10.1145/36611478:EICS(1-27)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1145/3661147
Gaspar-Figueiredo DFernández-Diego MNuredini RAbrahao SInsfran E(2024)Reinforcement Learning-Based Framework for the Intelligent Adaptation of User InterfacesCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661329(40-48)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3660515.3661329
Mo GDudley JChan LLiao YOulasvirta AKristensson P(2024)Cooperative Multi-Objective Bayesian Design OptimizationACM Transactions on Interactive Intelligent Systems10.1145/365764314:2(1-28)Online publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1145/3657643
Cai TNiklaus AKerr BKraley MBylinskii Z(2024)COR Themes for Readability from Iterative FeedbackProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642108(1-23)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642108
Hofmann MAuradkar NBirchfield JCao JHughes AKim GKurpad SLum KMack KNilakantan ASeehorn MWarnock EMankoff JHudson S(2023)OPTIMISM: Enabling Collaborative Implementation of Domain Specific Metaheuristic OptimizationProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580904(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580904
Wang WGai WZang CBao XYang C(2023)Personalized Recommendation of User Interfaces Based on Foot Interaction2023 IEEE Smart World Congress (SWC)10.1109/SWC57546.2023.10448805(1-8)Online publication date: 28-Aug-2023
https://doi.org/10.1109/SWC57546.2023.10448805
Vidmanov DAlfimtsev A(2023)MARLMUI: Multi-Agent Reinforcement Learning Approach in Mobile Adaptive User Interface2023 5th International Youth Conference on Radio Electronics, Electrical and Power Engineering (REEPE)10.1109/REEPE57272.2023.10086785(1-5)Online publication date: 16-Mar-2023
https://doi.org/10.1109/REEPE57272.2023.10086785
Lomas JLin ADikker SForster DLupetti MHuisman GHabekost JBeardow CPandey PAhmad NMiyapuram KMullen TCooper Pvan der Maden WCross E(2022)Resonance as a Design Strategy for AI and Social RobotsFrontiers in Neurorobotics10.3389/fnbot.2022.85048916Online publication date: 27-Apr-2022
https://doi.org/10.3389/fnbot.2022.850489
Nunes Vilaza GDoherty KMcCashin DCoyle DBardram JBarry M(2022)A Scoping Review of Ethics Across SIGCHIProceedings of the 2022 ACM Designing Interactive Systems Conference10.1145/3532106.3533511(137-154)Online publication date: 13-Jun-2022
https://dl.acm.org/doi/10.1145/3532106.3533511
Li ZZhao MDas DZHAO HMa YLiu WBeaudouin-Lafon MWang FRamakrishnan IBi X(2022)Select or Suggest? Reinforcement Learning-based Method for High-Accuracy Target Selection on TouchscreensProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517472(1-15)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3517472
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents