research-article

Public Access

Lessons Learned from Two Cohorts of Personal Informatics Self-Experiments

Authors:

Nediyana Daskalova,

Karthik Desingh,

Alexandra Papoutsaki,

Jeff HuangAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 1, Issue 3

Article No.: 46, Pages 1 - 22

https://doi.org/10.1145/3130911

Published: 11 September 2017 Publication History

Abstract

Self-experiments allow people to investigate their own individual outcomes from behavior change, often with the aid of personal tracking devices. The challenge is to design scientifically valid self-experiments that can reach conclusive results. In this paper, we aim to understand how novices run self-experiments when they are provided with a structured lesson in experimental design. We conducted a study on self-experimentation with two cohorts of students, where a total of 34 students performed a self-experiment of their choice. In the first cohort, students were given only two restrictions: a specific number of variables to track and a set duration for the study. The findings from this cohort helped us generate concrete guidelines for running a self-experiment, and use them as the format for the next cohort. A second cohort of students used these guidelines to conduct their own self-experiments in a more structured manner. Based on the findings from both cohorts, we propose a set of guidelines for running successful self-experiments that address the pitfalls encountered by students in the study, such as inadequate study design and analysis methods. We also discuss broader implications for future self-experimenters and designers of tools for self-experimentation.

References

[1]

2015. Fitbit. (2015). https://www.fitbit.com/

[2]

Shipra Agrawal and Navin Goyal. 2012. Analysis of Thompson Sampling for the Multi-armed Bandit Problem. In Proceedings of COLT. 39--1.

[3]

David H. Barlow, Nock K. Matthew, and Michel Hersen. 2008. Single case experimental designs: Strategies for studying behavior for change. Pearson.

[4]

Colin Barr, Maria Marois, Ida Sim, Christopher H. Schmid, Barth Wilsey, Deborah Ward, Naihua Duan, Ron D. Hays, Joshua Selsky, Joseph Servadio, and others. 2015. The PREEMPT study-evaluating smartphone-assisted n-of-1 trials in patients with chronic pain: study protocol for a randomized controlled trial. Trials 16, 1 (2015), 67.

[5]

Frank Bentley, Konrad Tollmar, Peter Stephenson, Laura Levy, Brian Jones, Scott Robertson, Ed Price, Richard Catrambone, and Jeff Wilson. 2013. Health Mashups: Presenting Statistical Patterns Between Wellbeing Data and Context in Natural Language to Promote Behavior Change. ACM Trans. Comput.-Hum. Interact. 20, 5, Article 30 (Nov. 2013), 27 pages.

Digital Library

[6]

Olivier Chapelle and Lihong Li. 2011. An empirical evaluation of thompson sampling. In Advances in neural information processing systems. 2249--2257.

Digital Library

[7]

Eun Kyoung Choe, Nicole B. Lee, Bongshin Lee, Wanda Pratt, and Julie A. Kientz. 2014. Understanding Quantified-selfers’ Practices in Collecting and Exploring Personal Data. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14). 1143--1152.

Digital Library

[8]

Nediyana Daskalova, Danaë Metaxa-Kakavouli, Adrienne Tran, Nicole Nugent, Julie Boergers, John McGeary, and Jeff Huang. 2016. SleepCoacher: A Personalized Automated Self-Experimentation System for Sleep Recommendations. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 347--358.

Digital Library

[9]

Joan Didion. 1968. On Keeping a Notebook. (1968). Retrieved September 24, 2015 from https://penusa.org/sites/default/files/didion.pdf.

[10]

Naihua Duan, Ian Eslick, N Gabler, Heather Kaplan, Richard Kravitz, Eric Larson, Wilson Pace, Christopher Schmid, Ida Sim, and Sunita Vohra. 2014. Design and Implementation of N-of-1 Trials: A User’s Guide. Agency for Healthcare Research and Quality. www.effectivehealthcare.ahrq.gov/N-1-Trials.cfm

[11]

Eugene S. Edgington. 1987. Randomized single-subject experiments and statistical tests. Journal of Counseling Psychology 34, 4 (1987), 437--442.

[12]

Garabed Eknoyan. 1999. Santorio Sanctorius (1561--1636)-founding father of metabolic balance studies. American journal of nephrology 19, 2 (1999), 226--233.

[13]

Daniel A. Epstein, An Ping, James Fogarty, and Sean A. Munson. 2015. A lived informatics model of personal informatics. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 731--742.

Digital Library

[14]

Deborah Estrin and Ida Sim. 2010. Open mHealth Architecture: An Engine for Health Care Innovation. Science 330, 6005 (2010), 759--760. arXiv:http://www.sciencemag.org/content/330/6005/759.full.pdf

[15]

BJ Fogg. 2015. Tiny Habits. (2015). Retrieved September 24, 2015 from http://tinyhabits.com/.

[16]

Martin J. Gardner and Douglas G. Altman. 1986. Confidence intervals rather than P values: estimation rather than hypothesis testing. Br Med J (Clin Res Ed) 292, 6522 (1986), 746--750.

[17]

Anne Germain, Robin Richardson, Douglas E Moul, Oommen Mammen, Gretchen Haas, Steven D Forman, Noelle Rode, Amy Begley, and Eric A Nofzinger. 2012. Placebo-controlled comparison of prazosin and cognitive-behavioral treatments for sleep disturbances in US Military Veterans. Journal of psychosomatic research 72, 2 (2012), 89--96.

[18]

Andria Hanbury, Katherine Farley, Carl Thompson, Paul M. Wilson, Duncan Chambers, and Heather Holmes. 2013. Immediate versus sustained effects: interrupted time series analysis of a tailored intervention. Implementation Science 8, 1 (2013), 130--147.

[19]

Eric B Hekler, Winslow Burleson, and Jisoo Lee. 2013. A DIY Self-Experimentation Toolkit for Behavior Change. In Personal Informatics in the Wild: Hacking Habits for Health 8 Happiness at the ACM-CHI Conference. ACM.

[20]

Mieke Heyvaert and Patrick Onghena. 2014. Randomization tests for single-case experiments: State of the art, state of the science, and state of the application. Journal of Contextual Behavioral Science 3, 1 (2014), 51--64.

[21]

Jim A. Horne and Olov Ostberg. 1976. A self-assessment questionnaire to determine morningness-eveningness in human circadian rhythms. International Journal of Chronobiology 4, 2 (1976), 97--110.

[22]

Hsiu-Fang Hsieh and Sarah E Shannon. 2005. Three approaches to qualitative content analysis. Qualitative health research 15, 9 (2005), 1277--1288.

[23]

Bradley E. Huitema, Ron Van Houten, and Hana Manal. 2014. Time-series intervention analysis of pedestrian countdown timer effects. Accident Analysis 8 Prevention 72 (2014), 23--31.

[24]

W Paul Jones. 2003. Single-case time series with Bayesian analysis: a practitioner’s guide. (Methods, Plainly Speaking). Measurement and evaluation in counseling and development 36, 1 (2003), 28--40.

[25]

Ravi Karkar, Jasmine Zia, Roger Vilardaga, Sonali R. Mishra, James Fogarty, Sean A. Munson, and Julie A. Kientz. 2015. A framework for self-experimentation in personalized health. Journal of the American Medical Informatics Association (2015), ocv150.

[26]

Matthew Kay, Gregory L Nelson, and Eric B Hekler. 2016. Researcher-centered design of statistics: Why Bayesian statistics better fit the culture and incentives of HCI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 4521--4532.

Digital Library

[27]

Thomas R. Kratochwill, John H. Hitchcock, Robert H. Horner, Joel R. Levin, Samuel L. Odom, David M. Rindskopf, and William R. Shadish. 2013. Single-Case Intervention Research Design Standards. Remedial and Special Education 34, 1 (2013), 26--38. arXiv:http://rse.sagepub.com/content/34/1/26.full.pdf+html

[28]

Quantified Self Labs. 2012. About the Quantified Self. (2012). Retrieved September 08, 2015 from http://quantifiedself.com/about/.

[29]

Gary P Latham. 2003. Goal Setting:: A Five-Step Approach to Behavior Change. Organizational Dynamics 32, 3 (2003), 309--318.

[30]

Jisoo Lee, Erin Walker, Winslow Burleson, Matthew Kay, Matthew Buman, and Eric B Hekler. 2017. Self-experimentation for behavior change: Design and formative evaluation of two approaches. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 6837--6849.

Digital Library

[31]

Ian Li, Anind Dey, and Jodi Forlizzi. 2010. A Stage-based Model of Personal Informatics Systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10). 557--566.

Digital Library

[32]

Allen Neuringer. 1981. Self-experimentation: A call for change. Behaviorism 9, 1 (1981), 79--94.

[33]

Gemma Phillips, Lambert Felix, Leandro Galli, Vikram Patel, and Philip Edwards. 2010. The effectiveness of M-health technologies for improving health and health services: a systematic review protocol. BMC Research Notes 3, 1 (2010), 250.

[34]

Seth Roberts. 2004. Self-experimentation as a source of new ideas: Ten examples about sleep, mood, health, and weight. Behavioral and Brain Sciences 27, 2 (2004), 227 -- 288.

[35]

Seth Roberts. 2010. The unreasonable effectiveness of my self-experimentation. Medical hypotheses 75, 6 (2010), 482--489.

[36]

Seth Roberts. 2012. The reception of my self-experimentation. Journal of Business Research 65, 7 (2012), 1060--1066.

[37]

Steven L Scott. 2010. A modern Bayesian look at the multi-armed bandit. Applied Stochastic Models in Business and Industry 26, 6 (2010), 639--658.

Digital Library

[38]

William R. Shadish, David M. Rindskopf, and Larry V. Hedges. 2008. The state of the science in the meta-analysis of single-case experimental designs. Evidence-Based Communication Assessment and Intervention 2, 3 (2008), 188--196.

[39]

Justin D. Smith. 2012. Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods 17, 4 (2012), 510--550.

[40]

John B. Todman and Pat Dugard. 2001. Single-case and small-n experimental designs: A practical guide to randomization tests. Psychology Press.

[41]

Allen B. Weisse. 2012. Self-experimentation and its role in medical research. Texas Heart Institute Journal 39, 1 (2012), 51--54.

[42]

Joseph Jay Williams, Juho Kim, Anna Rafferty, Samuel Maldonado, Krzysztof Z Gajos, Walter S Lasecki, and Neil Heffernan. 2016. Axis: Generating explanations at scale with learnersourcing and machine learning. In Proceedings of the Third (2016) ACM Conference on Learning@ Scale. ACM, 379--388.

Digital Library

Cited By

Loerakker MNiess JBentvelzen MWoźniak P(2024)Designing Data Visualisations for Self-Compassion in Personal InformaticsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314487:4(1-22)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631448
Jung GPark SLee U(2024)DeepStress: Supporting Stressful Context Sensemaking in Personal Informatics Systems Using a Quasi-experimental ApproachProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642766(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642766
Silva LCibrian FBonang CBhattacharya AMin AMonteiro EBeltran JSchuck SLakes KHayes GEpstein D(2024)Co-Designing Situated Displays for Family Co-Regulation with ADHD ChildrenProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642745(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642745
Show More Cited By

Index Terms

Lessons Learned from Two Cohorts of Personal Informatics Self-Experiments
1. Human-centered computing
  1. Ubiquitous and mobile computing

Recommendations

Looking for the unusual: how older adults utilize self-tracking techniques for health management
PervasiveHealth '16: Proceedings of the 10th EAI International Conference on Pervasive Computing Technologies for Healthcare

Self-tracking applications for health management have become popular both in industry and in academia. Half of older adults in the U.S. track health indicators, but they rarely use technology for that purpose. We conduct a qualitative study to ...
An Article a Day: Towards Personal Informatics for Healthy News Consumption
CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

Technological logs can enable us to self-reflect on our habits through data. Avid news consumers, who read many articles and take an interest in news media, may have a pronounced interest in tracking their news habits. From a lens of personal informatics,...
Self-tracking for Mental Wellness: Understanding Expert Perspectives and Student Experiences
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Previous research suggests an important role for self-tracking in promoting mental wellness. Recent studies with college student populations have examined the feasibility of collecting everyday mood, activity, and social data. However, these studies do ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 1, Issue 3

September 2017

2023 pages

EISSN:2474-9567

DOI:10.1145/3139486

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 September 2017

Accepted: 01 July 2017

Revised: 01 May 2017

Received: 01 February 2017

Published in IMWUT Volume 1, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
945
Total Downloads

Downloads (Last 12 months)145
Downloads (Last 6 weeks)30

Reflects downloads up to 09 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Loerakker MNiess JBentvelzen MWoźniak P(2024)Designing Data Visualisations for Self-Compassion in Personal InformaticsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314487:4(1-22)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631448
Jung GPark SLee U(2024)DeepStress: Supporting Stressful Context Sensemaking in Personal Informatics Systems Using a Quasi-experimental ApproachProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642766(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642766
Silva LCibrian FBonang CBhattacharya AMin AMonteiro EBeltran JSchuck SLakes KHayes GEpstein D(2024)Co-Designing Situated Displays for Family Co-Regulation with ADHD ChildrenProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642745(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642745
Xu KYan XRyu MNewman MArriaga R(2024)Understanding the Effect of Reflective Iteration on Individuals’ Physical Activity PlanningProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641937(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641937
Kabir KWiese J(2023)A Meta-Synthesis of the Barriers and Facilitators for Personal Informatics SystemsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108937:3(1-35)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610893
Murnane EGlazko YCosta JYao RZhao GMoya PLanday J(2023)Narrative-Based Visual Feedback to Encourage Sustained Physical ActivityProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35807867:1(1-36)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580786
Nakamura YNakaoka RMatsuda YYasumoto K(2023)eat2picProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35807847:1(1-23)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580784
Kwon SLee HPark SHeo YLee KKang Y(2023)‘Um, so like, is this how I speak?’: design implications for automated visual feedback systems on speechBehaviour & Information Technology10.1080/0144929X.2023.2271997(1-20)Online publication date: 9-Nov-2023
https://doi.org/10.1080/0144929X.2023.2271997
Rimbert SFleck S(2023)Long-term kinesthetic motor imagery practice with a BCIComputers in Human Behavior10.1016/j.chb.2023.107789146:COnline publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1016/j.chb.2023.107789
Lushnikova ABongard-Blanchy KKoenig VLallemand C(2023)Eliciting Meaningful Collaboration Metrics: Design Implications for Self-Tracking Technologies at WorkHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42286-7_36(643-664)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-42286-7_36
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents