research-article

Public Access

Developing and Validating the User Burden Scale: A Tool for Assessing User Burden in Computing Systems

Authors:

Nina Shahriaree,

Eric B. Hekler,

Julie A. KientzAuthors Info & Claims

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Pages 3988 - 3999

https://doi.org/10.1145/2858036.2858448

Published: 07 May 2016 Publication History

Abstract

Computing systems that place a high level of burden on their users can have a negative affect on initial adoption, retention, and overall user experience. Through an iterative process, we have developed a model for user burden that consists of six constructs: 1) difficulty of use, 2) physical, 3) time and social, 4) mental and emotional, 5) privacy, and 6) financial. If researchers and practitioners can have an understanding of the overall level of burden systems may be having on the user, they can have a better sense of whether and where to target future design efforts that can reduce those burdens. To help assist with understanding and measuring user burden, we have also developed and validated a measure of user burden in computing systems called the User Burden Scale (UBS), which is a 20-item scale with 6 individual sub-scales representing each construct. This paper presents the process we followed to develop and validate this scale for use in evaluating user burden in computing systems. Results indicate that the User Burden Scale has good overall inter-item reliability, convergent validity with similar scales, and concurrent validity when compared to systems abandoned vs. those still in use.

References

[1]

Sajay Arthanat, Stephen M. Bauer, James A. Lenker, Susan M. Nochajski, and Yow Wu B. Wu. 2007. Conceptualization and measurement of assistive technology usability. Disability & Rehabilitation: Assistive Technology 2, 4: 235--248.

[2]

Ilhan Aslan, Martin Murer, Verena Fuchsberger, Andrew Fugard, and Manfred Tscheligi. 2013. Workload on your fingertips: the influence of workload on touchbased drag and drop. In Proceedings of the 2013 ACM international conference on Interactive tabletops and surfaces (ITS '13), 417--420.

Digital Library

[3]

Sasikanth Avancha, Amit Baxi, and David Kotz. 2012. Privacy in mobile technology for personal healthcare. ACM Computing Surveys (CSUR) 45, 1: 3.

Digital Library

[4]

James E. Bailey and Sammy W. Pearson. 1983. Development of a tool for measuring and analyzing computer user satisfaction. Management science 29, 5: 530--545.

Digital Library

[5]

Louise Barkhuus and Anind K. Dey. 2003. LocationBased Services for Mobile Telephony: a Study of Users' Privacy Concerns. In INTERACT, 3: 702--712.

[6]

Thomas Beauvisage. 2009. Computer usage in daily life. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '09), 575--584.

Digital Library

[7]

John L. Bennett 1984. Managing to meet usability requirements: establishing and meeting software development goals. Visual display terminals, 161--84.

[8]

Mark Bilandzic. 2010. The embodied hybrid space: designing ubiquitous computing towards an amplification of situated real world experiences. In Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction (OzCHI '10), 422--427.

Digital Library

[9]

Richard E. Boyatzis. 1998. Transforming qualitative information: Thematic analysis and code development. Sage.

[10]

John Brooke. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194: 4--7.

[11]

Michael Buhrmester, Tracy Kwang, and Samuel D. Gosling. 2011. Amazon's Mechanical Turk a new source of inexpensive, yet high-quality, data?. Perspectives on psychological science 6, 1: 3--5.

[12]

Rudy Den Buurman. 1997. User-centred design of smart products. Ergonomics 40, 10: 1159--1169.

[13]

Fang Chen, Natalie Ruiz, Eric Choi, Julien Epps, M. Asif Khawaja, Ronnie Taib, Bo Yin, and Yang Wang. Multimodal behavior and interaction as indicators of cognitive load. ACM Transactions on Interactive Intelligent Systems (TiiS '12) 2, 4: 22.

Digital Library

[14]

John P. Chin, Virginia A. Diehl, and Kent L. Norman. 1988. Development of an instrument measuring user satisfaction of the human-computer interface. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '88), 213--218.

Digital Library

[15]

Lee J. Cronbach and R. L. Thorndike. 1971. Educational measurement. Test validation, 443--507.

[16]

Fred D. Davis. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 319--340.

Digital Library

[17]

Fred D. Davis. 1993. User acceptance of information technology: system characteristics, user perceptions and behavioral impacts. International journal of man machine studies 38, 3: 475--487.

Digital Library

[18]

Louise Demers, Rhoda Weiss-Lambrou and Bernadette Ska. 2002. The Quebec User Evaluation of Satisfaction with Assistive Technology (QUEST 2.0): an overview and recent progress. Technology and Disability 14, 3: 101--105.

[19]

David De Vaus. 2002. Analyzing social science data: 50 key problems in data analysis. Sage.

[20]

Judith Donath. 2014. How social media design shapes society. In Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems (CHI '14), 1057--1058.

Digital Library

[21]

Joseph S. Dumas and Janice Redish. 1999. A practical guide to usability testing. Intellect Books.

Digital Library

[22]

Günther Gediga, Kai-Christoph Hamborg, and Ivo Düntsch. 1999. The IsoMetrics usability inventory: an operationalization of ISO 9241--10 supporting summative and formative evaluation of software systems. Behaviour & Information Technology 18, 3: 151--164.

[23]

Joseph A. Gliem and Rosemary R. Gliem. 2003. Calculating, interpreting, and reporting Cronbach's alpha reliability coefficient for Likert-type scales. Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education.

[24]

John D. Gould and Clayton Lewis. 1985. Designing for usability: key principles and what designers think. Communications of the ACM 28, 3: 300--311.

Digital Library

[25]

Jeroen B. Guinée. 2002. Handbook on life cycle assessment operational guide to the ISO standards." The international journal of life cycle assessment 7, 5: 311--313.

[26]

Robert Harmon, David Raffo, and Stuart Faulk. 2003. Incorporating price sensitivity measurement into the software engineering process. In Management of Engineering and Technology (PICMET'03), Technology Management for Reshaping the World. Portland International Conference on, 316--323.

[27]

Sandra G. Hart and Lowell E. Staveland. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology, 52: 139--183.

[28]

Leanne M. Hirshfield, Erin Treacy Solovey, Audrey Girouard, James Kebinger, Robert JK Jacob, Angelo Sassaroli, and Sergio Fantini. 2009. Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 2185--2194.

Digital Library

[29]

Jason I. Hong, Jennifer D. Ng, Scott Lederer, and James A. Landay. 2004. Privacy risk models for designing privacy-sensitive ubiquitous computing systems. In Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques (DIS '04), 91--100.

Digital Library

[30]

Salar Jahedi and Fabio Méndez. 2014. On the advantages and disadvantages of subjective measures. Journal of Economic Behavior & Organization 98: 97114.

[31]

Beom Suk Jin and Yong Gu Ji. 2010. Usability risk level evaluation for physical user interface of mobile phone. Computers in Industry 61, 4: 350--363.

Digital Library

[32]

Jurek Kirakowski and A. Dillon. 1988. The computer user satisfaction inventory (CUSI): Manual and scoring key. Cork, Ireland: Human Factors Research Group, University College of Cork.

[33]

Jurek Kirakowski and Mary Corbett. 1993. SUMI: The software usability measurement inventory. British journal of educational technology 24, 3: 210--212.

[34]

Jonathan Klein, Youngme Moon, and Rosalind W. Picard. 2002. This computer responds to user frustration:: Theory, design, and results. Interacting with computers 14, no. 2 (2002): 119--140.

[35]

Shelah Leader, Phillip Jacobson, James Marcin, Ralph Vardis, Mark Sorrentino, and Dennis Murray. 2002. A method for identifying the financial burden of hospitalized infants on families. Value in Health 5, 1: 55--59.

[36]

Han X. Lin, Yee-Yin Choong, and Gavriel Salvendy. 1997. A proposed index of usability: a method for comparing the relative usability of different software systems. Behaviour & information technology 16, 4--5: 267277.

[37]

Jakob Nielsen. 1991. Usability metrics and methodologies. ACM SIGCHI Bulletin 23, 2: 37--39.

Digital Library

[38]

Jum C. Nunnally, Ira H. Bernstein, and Jos MF ten Berge. 1967. Psychometric theory. Vol. 226. NY: McGraw-Hill.

[39]

Jane Ogden and Jessica Lo. 2012. How meaningful are data from Likert scales? An evaluation of how ratings are made and the role of the response shift in the socially disadvantaged. Journal of health psychology 17, 3: 350--361.

[40]

Helen Petrie and Nigel Bevan. 2009. The evaluation of accessibility, usability and user experience. The universal access handbook: 10--20.

[41]

Ahmad Rahmati, Chad Tossell, Clayton Shepard, Philip Kortum, and Lin Zhong. 2012. Exploring iPhone usage: the influence of socioeconomic differences on smartphone adoption, usage and usability. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services (Mobile HCI '12), 11--20.

Digital Library

[42]

Jeffrey Rubin and Dana Chisnell. 2008. Handbook of usability testing: how to plan, design and conduct effective tests. John Wiley & Sons.

Digital Library

[43]

Jocelyn Scheirer, Raul Fernandez, Jonathan Klein, and Rosalind W. Picard. 2002. Frustrating the user on purpose: a step toward building an affective computer. Interacting with computers 14, 2: 93--118.

[44]

Brian Shackel. 1984. The concept of usability. Visual display terminals: Usability issues and health concerns: 45--87.

[45]

Ben Shneiderman. 1992. Designing the user interface: strategies for effective human-computer interaction. Vol. 3. Reading, MA: Addison-Wesley.

Digital Library

[46]

Deb Sledgianowski and Songpol Kulviwat. 2008. Social network sites: antecedents of user adoption and usage. Americas Conference on Information Systems (AMCIS) Proceedings: 83.

[47]

Godwin J. Udo. 2001. Privacy and security concerns as major barriers for e-commerce: a survey study. Information Management & Computer Security 9, 4: 165--174.

[48]

Robert S. Weiss. 1995. Learning from strangers: The art and method of qualitative interview studies. Simon and Schuster.

[49]

Anna M. Wichansky. 2000. Usability testing in 2000 and beyond. Ergonomics 43, 7: 998--1006.

[50]

David Wright. 2011. Should privacy impact assessments be mandatory?. Communications of the ACM 54, 8: 121131.

Digital Library

[51]

Svetlana Yarosh, Panos Markopoulos, and Gregory D. Abowd. 2014. Towards a questionnaire for measuring affective benefits and costs of communication technologies. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW '14), 84--96.

Digital Library

Cited By

Golbus JJeganathan VStevens REkechukwu WFarhan ZContreras RRao NTrumpower BBasu TLuff ESkolarus LNewman MNallamothu BDorsch M(2024)A Physical Activity and Diet Just‐in‐Time Adaptive Intervention to Reduce Blood Pressure: The myBPmyLife Study Rationale and DesignJournal of the American Heart Association10.1161/JAHA.123.03123413:2Online publication date: 16-Jan-2024
https://doi.org/10.1161/JAHA.123.031234
Loerakker MNiess JWoźniak P(2024)Technology which Makes You ThinkProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596158:2(1-24)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659615
Park H(2024)Re-examining User Burden in Human-AI Interaction: Focusing on a Domain-Specific ApproachExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638186(1-4)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3638186
Show More Cited By

Index Terms

Developing and Validating the User Burden Scale: A Tool for Assessing User Burden in Computing Systems
1. Human-centered computing

Recommendations

User Burden of Microinteractions: An In-lab Experiment Examining User Performance and Perceived Burden Related to In-situ Self-reporting
MobileHCI '21: Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction

In-situ self-reporting on smartwatches has been widely-used to collect data about participants’ experience. Yet it places a burden on participants by requiring immediate response in context. Such user burden can be studied through users’ performance and ...
Developing practical tools for user experience evaluation: a case from mobile news journalism
ECCE '09: European Conference on Cognitive Ergonomics: Designing beyond the Product --- Understanding Activity and User Experience in Ubiquitous Environments

We present a questionnaire called Attrak-Work to support the evaluation of user experience of mobile systems in the context of mobile news journalism. We discuss theoretical background of the questionnaire and describe the development process including ...
Actionable Inexpensive Games User Research
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems

This course will allow people to understand the intricacies of rapid games user research methods. For this we will weave together playtesting exercises and help participants turn player feedback into actionable design recommendations. The course is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

May 2016

6108 pages

ISBN:9781450333627

DOI:10.1145/2858036

General Chairs:
Jofish Kaye
Yahoo
,
Allison Druin
University of Maryland / National Park Service
,
Program Chairs:
Cliff Lampe
University of Michigan
,
Dan Morris
Microsoft
,
Juan Pablo Hourcade
University of Iowa

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

CHI'16

Sponsor:

SIGCHI

CHI'16: CHI Conference on Human Factors in Computing Systems

May 7 - 12, 2016

California, San Jose, USA

Acceptance Rates

CHI '16 Paper Acceptance Rate 565 of 2,435 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

90
Total Citations
View Citations
2,936
Total Downloads

Downloads (Last 12 months)626
Downloads (Last 6 weeks)75

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Golbus JJeganathan VStevens REkechukwu WFarhan ZContreras RRao NTrumpower BBasu TLuff ESkolarus LNewman MNallamothu BDorsch M(2024)A Physical Activity and Diet Just‐in‐Time Adaptive Intervention to Reduce Blood Pressure: The myBPmyLife Study Rationale and DesignJournal of the American Heart Association10.1161/JAHA.123.03123413:2Online publication date: 16-Jan-2024
https://doi.org/10.1161/JAHA.123.031234
Loerakker MNiess JWoźniak P(2024)Technology which Makes You ThinkProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596158:2(1-24)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659615
Park H(2024)Re-examining User Burden in Human-AI Interaction: Focusing on a Domain-Specific ApproachExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3638186(1-4)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3638186
Slovak PMunson S(2024)HCI Contributions in Mental Health: A Modular Framework to Guide Psychosocial Intervention DesignProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642624(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642624
Zhang H(2024)Searching for the Non-Consequential: Dialectical Activities in HCI and the Limits of ComputersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641945(1-13)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641945
Plotkina LSri Ramalu S(2024)Unearthing AI coaching chatbots capabilities for professional coaching: a systematic literature reviewJournal of Management Development10.1108/JMD-06-2024-018243:6(833-848)Online publication date: 26-Sep-2024
https://doi.org/10.1108/JMD-06-2024-0182
Ro YKwon B(2024)Does User Burden Matter?: Age Differences in User Behavior of Self-Service TechnologyInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2375085(1-20)Online publication date: 9-Aug-2024
https://doi.org/10.1080/10447318.2024.2375085
Horvath MPittman BO’Malley SGrutman AKhan NGueorguieva RBrewer JGarrison K(2024)Smartband-based smoking detection and real-time brief mindfulness intervention: findings from a feasibility clinical trialAnnals of Medicine10.1080/07853890.2024.235280356:1Online publication date: Jun-2024
https://doi.org/10.1080/07853890.2024.2352803
Yfantidou SYfantidou GBalaska PVakali A(2024)Negotiation strategies in ubiquitous human-computer interaction: a novel storyboards scale & field studyMultimedia Tools and Applications10.1007/s11042-024-20240-9Online publication date: 19-Sep-2024
https://doi.org/10.1007/s11042-024-20240-9
D’Silva AHua NModayil MSeidel JMarshall D(2024)Digital Health Interventions Are Effective for Irritable Bowel Syndrome Self-Management: A Systematic ReviewDigestive Diseases and Sciences10.1007/s10620-024-08672-7Online publication date: 14-Oct-2024
https://doi.org/10.1007/s10620-024-08672-7
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents