skip to main content
10.1145/3290605.3300435acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Reading Face, Reading Health: Exploring Face Reading Technologies for Everyday Health

Published: 02 May 2019 Publication History

Abstract

With the recent advancement in computer vision, Artificial Intelligence (AI), and mobile technologies, it has become technically feasible for computerized Face Reading Technologies (FRTs) to learn about one's health in everyday settings. However, how to design FRT-based applications for everyday health practices remains unexplored. This paper presents a design study with a technology probe called Faced, a mobile health checkup application based on the facial diagnosis method from Traditional Chinese Medicine (TCM). A field trial of Faced with 10 participants suggests potential usage modes and highlights a number of critical design issues in the use of FRTs for everyday health, including adaptability, practicality, sensitivity, and trustworthiness. We end by discussing design implications to address the unique challenges of fully integrating FRTs into everyday health practices.

Supplementary Material

MP4 File (paper205p.mp4)
Preview video

References

[1]
Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y. Lim, and Mohan Kankanhalli. 2018. Trends and Trajectories for Explainable, Accountable and Intelligible Systems: An HCI Research Agenda. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 582, 18 pages.
[2]
Yasmina Andreu-Cabedo, Pedro Castellano, Sara Colantonio, Giuseppe Coppini, Riccardo Favilla, Danila Germanese, Giorgos Giannakakis, Daniela Giorgi, Marcus Larsson, Paolo Marraccini, et al. 2015. Mirror mirror on the wall--An intelligent multisensory mirror for well-being self-assessment. In Multimedia and Expo (ICME), 2015 IEEE International Conference on. IEEE, 1--6.
[3]
Amid Ayobi, Paul Marshall, Anna L Cox, and Yunan Chen. 2017. Quantifying the body and caring for the mind: self-tracking in multiple sclerosis. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 6889--6901.
[4]
Hilary Browne, Benjamin B. Bederson, Catherine Plaisant, and Allison Druin. 2001. Designing an Interactive Message Board as a Technology Probe for Family Communication. (2001).
[5]
Felix Cheung. 2011. TCM: Made in China. Nature 480, 7378 (2011).
[6]
James Clawson, Jessica A. Pater, Andrew D. Miller, Elizabeth D. Mynatt, and Lena Mamykina. 2015. No longer wearing:investigating the abandonment of personal health-tracking technologies on craigslist. In ACM International Joint Conference on Pervasive and Ubiquitous Computing. 647--658.
[7]
Gari D Clifford. 200 Shortliffe Edward H, Cimino James J: "Biomedical Informatics; Computer Applications in Health Care and Biomedicine". Biomedical Engineering Online 5, 1 (2006), 61--61.
[8]
Giuseppe Coppini, Veronica Chiara Zuccaia, Renata De Marià, JulieAnne Nazare, Maria Aurora Morales, and Sara Colantonio. 2017. User acceptance of self-monitoring technology to prevent cardio-metabolic diseases: The Wize Mirror. In Wireless and Mobile Computing, Networking and Communications (WiMob),. IEEE, 265--271.
[9]
Ciprian Adrian Corneanu, Marc Oliu Simón, Jeffrey F Cohn, and Sergio Escalera Guerrero. 2016. Survey on rgb, 3d, thermal, and multimodal approaches for facial expression recognition: History, trends, and affect-related applications. IEEE transactions on pattern analysis and machine intelligence 38, 8 (2016), 1548--1568.
[10]
Henriette Cramer, Vanessa Evers, Satyan Ramlal, Maarten van Someren, Lloyd Rutledge, Natalia Stash, Lora Aroyo, and Bob Wielinga. 2008. The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-Adapted Interaction 18, 5 (20 Aug 2008), 455.
[11]
Daniel A. Epstein, Monica Caraway, Chuck Johnston, An Ping, James Fogarty, and Sean A. Munson. 2016. Beyond Abandonment to Next Steps:Understanding and Designing for Life after Personal Informatics Tool Use. 2016 (2016), 1109--1113.
[12]
Motahhare Eslami, Aimee Rickman, Kristen Vaccaro, Amirhossein Aleyasen, Andy Vuong, Karrie Karahalios, Kevin Hamilton, and Christian Sandvig. 2015. "I Always Assumed That I Wasn'T Really That Close to {Her}": Reasoning About Invisible Algorithms in News Feeds. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 153--162.
[13]
Alexander G Logan et al. 2007. Mobile phone-based remote patient monitoring system for management of hypertension in diabetic patients. American Journal of Hypertension 20, 9 (2007), 942--948.
[14]
Darren L Walters et al. 2010. A mobile phone-based care model for outpatient cardiac rehabilitation: the care assessment platform (CAP). BMC Cardiovascular Disorders 10, 1 (2010), 5--5.
[15]
Dena M Bravata et al. 2007. Using Pedometers to Increase Physical Activity and Improve Health: A Systematic Review. JAMA 298, 19 (2007), 2296--2304.
[16]
Felicia Cordeiro et al. 2015. Barriers and Negative Nudges: Exploring Challenges in Food Journaling. 2015 (2015), 1159--1162.
[17]
Francisco Nunes et al. 2015. Self-Care Technologies in HCI: Trends, Tensions, and Opportunities. ACM Transactions on Computer-Human Interaction 22, 6 (2015), 33.
[18]
Jakob E Bardram et al. 2013. Designing mobile health technology for bipolar disorder: a field trial of the monarca system. (2013), 2627-- 2636.
[19]
James Jengweei Lin et al. 2006. Fish'n'Steps: encouraging physical activity with an interactive computer game. ubiquitous computing (2006), 261--278.
[20]
Joyce K Anastasi et al. 2014. Tongue Inspection in TCM: Observations in a Study Sample of Patients Living with HIV. Medical Acupuncture 26, 1 (2014), 15--22.
[21]
Lena Mamykina et al. 2008. MAHI: investigation of social scaffolding for reflective thinking in diabetes management. (2008), 477--486.
[22]
Rui Guo et al. 2015. Analysis and Recognition of Traditional Chinese Medicine Pulse Based on the Hilbert-Huang Transform and Random Forest in Patients with Coronary Heart Disease. Evidence-based Complementary and Alternative Medicine 2015 (2015), 895749--895749.
[23]
Gaver, Bill, Dunne, Tony, Pacenti, and Elena. 1999. Design: Cultural probes. Interactions 6, 1 (1999), 21--29.
[24]
Erik Gronvall and Nervo Verdezoto. 2013. Beyond self-monitoring: understanding non-functional aspects of home-based healthcare technology. (2013), 587--596.
[25]
M. Shamim Hossain and Ghulam Muhammad. 2015. Cloud-Assisted Speech and Face Recognition Framework for Health Monitoring. Mobile Networks Applications 20, 3 (2015), 391--399.
[26]
Yangyang Hu, Hong Lu, Jinhua Cheng, Wenqiang Zhang, Fufeng Li, and Weifei Zhang. 2016. Robust Lip Segmentation Based on Complexion Mixture Model. In Pacific-Rim Conference on Advances in Multimedia Information Processing. 85--94.
[27]
Hilary Hutchinson, Wendy Mackay, Westerlund Bo, Benjamin B. Bederson, Allison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Helen Evans, Heiko Hansen, and Nicolas Roussel. 2003. Technology probes: inspiring design for and with families. In Sigchi Conference on Human Factors in Computing Systems. 17--24.
[28]
H. Hutchinson, C. Plaisant, and A. Druin. 2002. Case Study: A Message Board as a Technology Probe for Family Communication and Coordination. (2002).
[29]
Yuma Inagawa, Junki Hakamta, and Masataka Tokumaru. 2013. A Support System for Healthy Eating Habits: Optimization of Recipe Retrieval. Springer Berlin Heidelberg. 168--172 pages.
[30]
Brielle Jaekel. {n. d.}. Sephora boosts augmented reality shopping with real-time facial recognition. http://t.cn/EGjuPZK. {Last accessed: August 29, 2018}.
[31]
Matthew Kay, Eun Kyoung Choe, Jesse Shepherd, Benjamin Greenstein, Nathaniel Watson, Sunny Consolvo, and Julie A Kientz. 2012. Lullaby: a capture & access system for understanding the sleep environment. In Proceedings of the 2012 ACM conference on ubiquitous computing. ACM, 226--234.
[32]
K Kleisner, V Chvátalová, and J Flegr. 2014. Perceived Intelligence Is Associated with Measured Intelligence in Men but Not Women. Plos One 9, 3 (2014), e81237.
[33]
Geoff Langdale, Judy Kay, and Bob Kummerfeld. 2006. Using an intergenerational communications system as a 'light-weight' technology probe. In Extended Abstracts Proceedings of the 2006 Conference on Human Factors in Computing Systems, CHI 2006, Montréal, Québec, Canada, April. 1001--1006.
[34]
Fang Li, Maylor K. H. Leung, Namisha Khanna, and Purva Satija. 2007. Online fortune telling system using biometrics feature. In IEEE International Conference on Systems, Man and Cybernetics. 4002--4007.
[35]
Y. M. Li, H. Z. Yang, W. B. Guan, Q. S. Ke, M. Dai, H. P. Xie, and S. J. Zhang. 2008. Therapeutic effect of traditional Chinese medicine on coagulation disorder and accompanying intractable jaundice in hepatitis B virus-related liver cirrhosis patients. World Journal of Gastroenterology 14, 39 (2008), 6060--6064.
[36]
Chang Liu, Changbo Zhao, Guozheng Li, Fufeng Li, and Zhi Wang. 2014. Computerized color analysis for facial diagnosis in traditional Chinese medicine. In IEEE International Conference on Bioinformatics and Biomedicine. 613--614.
[37]
Deborah Lupton. 20 Self-tracking, health and medicine. Health Sociology Review 26, 1 (2017), 1--5.
[38]
Andrew D Miller and Elizabeth D Mynatt. 2014. StepStream: a schoolbased pervasive social fitness system for everyday adolescent health. (2014), 2823--2832.
[39]
Francisco Nunes and Geraldine Fitzpatrick. 2018. Understanding the Mundane Nature of Self-care: Ethnographic Accounts of People Living with Parkinson's. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 402.
[40]
Kingsley C. Nwosu. 2016. MOBILE FACIAL RECOGNITION SYSTEM FOR PATIENT IDENTIFICATION IN MEDICAL EMERGENCIES FOR DEVELOPING ECONOMIES. Journal for the Advancement of Developing Economies 5, 1 (2016), 10.
[41]
William T. Odom, Abigail J. Sellen, Richard Banks, David S. Kirk, Tim Regan, Mark Selby, Jodi L. Forlizzi, and John Zimmerman. 2014. Designing for slowness, anticipation and re-visitation:a long term field study of the photobox. In ACM Conference on Human Factors in Computing Systems. 1961--1970.
[42]
Institute of Medicine (US) Committee on Health Literacy, L NielsenBohlman, A. M. Panzer, and D. A. Kindig. 2004. Health Literacy: A Prescription to End Confusion. National Academies Press 15, 4 (2004), 389--395.
[43]
Aisling Ann Okane, Yvonne Rogers, and Ann Blandford. 2015. Concealing or Revealing Mobile Medical Devices?: Designing for Onstage and Offstage Presentation. (2015), 1689--1698.
[44]
Michel Owayjan, Amer Dergham, Gerges Haber, Nidal Fakih, Ahmad Hamoush, and Elie Abdo. 2013. Face Recognition Security System. In International Joint Conferences on Computer, Information, and Systems Sciences, and Engineering.
[45]
A. Pantazi, A. Sebastian, T. A. Antonakopoulos, and P. Bächtold. 2008. Probe-based ultrahigh-density storage technology. Ibm Journal of Research Development 52, 4.5 (2008), 493--511.
[46]
Enrica Papi, Athina Belsi, and Alison H McGregor. 2015. A knee monitoring device and the preferences of patients living with osteoarthritis: a qualitative study. BMJ open 5, 9 (2015), e007980.
[47]
Stephen Purpura, Victoria Schwanda, Kaiton Williams, William Stubler, and Phoebe Sengers. 2011. Fit4life: the design of a persuasive technology promoting healthy behavior and ideal weight. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 423--432.
[48]
Wang Qi. 2002. Status and Prospect of Constitutional Theory in Traditional Chinese Medicine. China Journal of Basic Medicine in Traditional Chinese Medicine (2002).
[49]
A Radovic, C Farris, K Reynolds, E. C. Reis, E Miller, and B. D. Stein. 2014. Primary care providers' beliefs about teen and parent barriers to depression care. Journal of Developmental Behavioral Pediatrics Jdbp 35, 8 (2014), 534--8.
[50]
Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. FaceNet: A unified embedding for face recognition and clustering. (2015), 815--823.
[51]
Leon F Seltzer. {n. d.}. What Does It Mean When We Stick Our Tongues Out? http://t.cn/EGjudme. {Last accessed: August 31, 2018}.
[52]
J. J. Shu and Y. Sun. 2007. Developing classification indices for Chinese pulse diagnosis. Complementary Therapies in Medicine 15, 3 (2007), 190--198.
[53]
Aneesha Singh, Nadia Bianchi-Berthouze, and Amanda CdeC Williams. 2017. Supporting everyday function in chronic pain using wearable technology. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 3903--3915.
[54]
T Tempark and T Shwayder. 2012. Chinese fortune-telling based on face and body mole positions: a hidden agenda regarding mole removal. Archives of Dermatology 148, 6 (2012), 772.
[55]
Tammy Toscos and Kay Connelly. 2013. Designing for Positive Health Affect: Decoupling Negative Emotion and Health Monitoring Technologies. (2013), 153--160.
[56]
Hitomi Tsujita and Jun Rekimoto. 2011. Smiling makes happier:enhancing positive mood and communication with smileus encouraging digital appliances. In International Conference on Ubiquitous Computing. 1--10.
[57]
Jhim Kiel M. Verame, Enrico Costanza, and Sarvapali D. Ramchurn. 2016. The Effect of Displaying System Confidence Information on the Usage of Autonomous Systems for Non-specialist Applications: A Lab Study. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 4908--4920.
[58]
Yonggang Wang, Yue Zhou, Jie Yang, and Qing Xu. 200 An Image Analysis System for Tongue Diagnosis in Traditional Chinese Medicine. 3314 (2004), 1181--1186.
[59]
Kiyoshi Yasuda, Noriaki Kuwahara, and Kazunari Morimoto. 2009. Remote reminiscence talking and scheduling prompter for individuals with dementia using video phone. In International Conference on Universal Access in Human-Computer Interaction. Springer, 429--438.
[60]
Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks. IEEE Signal Processing Letters 23, 10 (2016), 1499-- 1503.
[61]
C. Zhao, G. Z. Li, F. Li, Z. Wang, and C. Liu. 2014. Qualitative and quantitative analysis for facial complexion in traditional Chinese medicine. Biomed Research International 2014, 3 (2014), 207589.
[62]
Jin Haia Zhou, Taoa Yang, Da Qinga Shen, and Xu Dongb Wang. 2010. Construction and application of eight principal syndrome differentiation knowledge base of tongue diagnosis based on ANN. Application Research of Computers 01, 2 (2010), 110--113.
[63]
??? and ???. 2014. ???????????? (Study on Face Shape of Patients with Five-Zang Organs Diseases). In ??? ???????????????????????.
[64]
???, ???, ???, ???, ??, ???, ???, ??, and ???. 2018. ?????????????????"???? ??????? (Study of Yunzhongyi Mobile Health Management Platform Based on Tongue,Face and Asking Diagnosis Information). ??????? 2 (2018).
[65]
??? and ???. 2015. ????????????????? ???? (Adaptive Facial Skin Region Extraction Based on Lip Color and Complexion). ?????? 31, 8 (2015), 1--3.

Cited By

View all

Index Terms

  1. Reading Face, Reading Health: Exploring Face Reading Technologies for Everyday Health

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
    May 2019
    9077 pages
    ISBN:9781450359702
    DOI:10.1145/3290605
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 May 2019

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. design study
    2. everyday healthcare
    3. face reading technologies
    4. facial diagnosis
    5. health
    6. health monitoring
    7. self care
    8. tcm
    9. traditional chinese medicine
    10. wellbeing

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    CHI '19
    Sponsor:

    Acceptance Rates

    CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)116
    • Downloads (Last 6 weeks)9
    Reflects downloads up to 03 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media