research-article

MoodScope: building a mood sensor from smartphone usage patterns

Authors:

Robert LiKamWa,

Nicholas D. Lane,

Lin ZhongAuthors Info & Claims

MobiSys '13: Proceeding of the 11th annual international conference on Mobile systems, applications, and services

Pages 389 - 402

https://doi.org/10.1145/2462456.2464449

Published: 25 June 2013 Publication History

Abstract

We report a first-of-its-kind smartphone software system, MoodScope, which infers the mood of its user based on how the smartphone is used. Compared to smartphone sensors that measure acceleration, light, and other physical properties, MoodScope is a "sensor" that measures the mental state of the user and provides mood as an important input to context-aware computing. We run a formative statistical mood study with smartphone-logged data collected from 32 participants over two months. Through the study, we find that by analyzing communication history and application usage patterns, we can statistically infer a user's daily mood average with an initial accuracy of 66%, which gradu-ally improves to an accuracy of 93% after a two-month personal-ized training period. Motivated by these results, we build a service, MoodScope, which analyzes usage history to act as a sensor of the user's mood. We provide a MoodScope API for developers to use our system to create mood-enabled applications. We further create and deploy a mood-sharing social application.

References

[1]

J. A. Russell. A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), pp. 1161--1178, 1980.

[2]

C. D. Batson, L. L. Shaw and K. C. Oleson. Distinguishing affect, mood and emotion: toward functionally-based conceptual distinctions. Review of Personality and Social Psychology, 13, Newbury Park, 1992.

[3]

P. Ekman. Universal and cultural differences in facial expressions of emotion. In Proc. Nebraska Symp. Motivation, 1971.

[4]

S. S. Tomkins. Affect, Imagery, Consciousness. Springer, 1962.

[5]

J. A. Russell. Affect grid: a single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57(3), 1989.

[6]

J. Tao and T. Tan. Affective computing: a review. Affective Computing and Intelligent Interaction, LNCS 3784, Springer, 2005.

Digital Library

[7]

M. Yik, Studying Affect among the Chinese: The circular way. Journal of Personality Assessment, 91(5), 2009.

[8]

K. Church, E. Hoggan and N. Oliver. A study of mobile mood awareness and communication through MobiMood. In Proc. NordiCHI, 2010.

Digital Library

[9]

A. Gluhak, M. Presser, L. Zhu, S. Esfandiyari and S. Kupschick. Towards mood based mobile services and applica-tions. In Proc. European Conference on Smart Sensing & Context (EuroSSC), 2007.

Digital Library

[10]

D. Watson, L.A. Clark, and A. Tellegen. Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, pages 1063--1070, 1988.

[11]

J. R. Crawford and J. D. Henry. The positive and negative affect schedule (PANAS): construct validity, measurement properties and normative data in a large non-clinical sample. British Journal of Clinical Psychology, 43, pages 245--265, 2004.

[12]

J. T. Larsen, A. P. McGraw and J. T. Cacioppo, Can people feel happy and sad at the same time? Journal of Personality and Social Psychology, 81(4), 1988.

[13]

Z. Zeng, M. Pantic, G. I. Roisman and T. S. Huang. A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1), 2009.

Digital Library

[14]

A. Vinciarelli, M. Pantic and H. Bourlard. Social signal processing: survey of an emerging domain. Image and Vision Computing, 2009.

Digital Library

[15]

J. F. Cohn. Foundations of human computing: Facial expression and emotion. In Proc. ACM Int'l Conf. Multimodal Inter-faces (ICMI), 2006.

Digital Library

[16]

M. Pantic and M.S. Bartlett. Machine analysis of facial expressions. Face Recognition, K. Delac and M. Grgic, eds., pages 377--416, I-Tech Education and Publishing, 2007.

[17]

P. Ekman, W.V. Friesen, and J.C. Hager. Facial action coding system. A Human Face, 2002.

[18]

C.M. Lee and S.S. Narayanan. Toward detecting emotions in spoken dialogs. IEEE Trans. Speech and Audio Processing, 13(2), pages 293--303, 2005.

[19]

B. Schuller, R.J. Villar, G. Rigoll, and M. Lang. Meta-classifiers in acoustic and linguistic feature fusion-based affect recognition. In Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing (ICASSP '05), pages 325--328, 2005.

[20]

R. W. Picard, Affective computing, MIT Press, 1997.

Digital Library

[21]

The Affectiva Q sensor, http://www.affectiva.com/q-sensor/.

[22]

A. Gluhak, M. Presser, L. Zhu, S. Esfandiyari and S. Kupschick. Towards mood based mobile services and applications. In Proc. European Conference on Smart Sensing & Context (EuroSSC), 2007.

Digital Library

[23]

K. K. Rachuri, M. Musolesi, C. Mascolo, P. J. Rentfrow, C. Longworth and A. Aucinas. EmotionSense: a mobile phones based adaptive platform for experimental social psychology research. In Proc. ACM Int'l Conf. on Ubiquitous Computing (UbiComp), 2010.

Digital Library

[24]

H. Lee, Y. Choi, S. Lee and I. P. Park. Towards unobtrusive emotion recognition for affective social communication. In Proc. IEEE Consumer Communications & Networking Conference (CCNC), 2012.

[25]

The mappiness application, http://www.mappiness.org.uk/.

[26]

The gottaFeeling application, http://gottafeeling.com/.

[27]

C. Shepard, A. Rahmati, C. Tossell, L. Zhong and P. Kortum. LiveLab: measuring wireless networks and smartphone users in the field. ACM SIGMETRICS Perform. Eval. Rev., 38(3), 2010.

Digital Library

[28]

J. P. Forgas, G. H. Bower and S.E. Krantz. The influence of mood on perceptions of social interactions, Journal of Experimental Social Psychology, 20(6), 1984.

[29]

L. A. Clark and D. Watson. Mood and the mundane: Rela-tions between daily life events and self-reported mood. Journal of Personality and Social Psychology, 54(2), 1988.

[30]

M. Ester, H.-P. Kriegel, J. Sander and X. Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proc. Second International Conference on Knowledge Discovery and Data Mining, 1996.

Digital Library

[31]

C. J. Beedie, P. C. Terry and A. M. Lane. Distinctions between emotion and mood. Cognition and Emotion, 19(6), 2005.

[32]

Hernandez, Javier, et al. "Mood meter: counting smiles in the wild." In Proc. ACM Int'l Conf. on Ubiquitous Computing (UbiComp), 2012.

Digital Library

[33]

G. Chittaranjan, J. Blom and D. Gatica-Perez. Who's who with Big-Five: Analyzing and Classifying personality traits with smartphones. In Proc. Int'l Symposium on Wearable Computers (ISWC), 2011.

Digital Library

[34]

L. Lu, D. Liu and H. Zhang. Automatic mood detection and tracking of music audio signals. IEEE Transactions on Audio, Speech and Language Processing, 14(1), 2006.

Digital Library

[35]

N. Japkowicz and S. Stephen. The class imbalance problem: A systematic study. Intelligent Data Analysis, 6(5), 2002.

Digital Library

[36]

A. Jain and D. Zongker. Feature selection: Evaluation, application, and small sample performance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(2), 1997.

Digital Library

[37]

A. Narayanan and V. Shmatikov. Myths and fallacies of Personally Identifiable Information. From Commun. ACM 53(6), pages 24--26, June 2010.

Digital Library

[38]

N. D. Lane, Y. Xu, H. Lu, S. Hu, T. Choudhury, A. T. Camp-bell and F. Zhao. Enabling large-scale human activity inference on smartphones using community similarity networks (CSN). In Proc. ACM Int'l Conf. on Ubiquitous Computing (UbiComp), 2011.

Digital Library

[39]

R. LiKamWa, Y. Liu, N. D. Lane and L. Zhong. "Can your smartphone infer your mood?", In Proc. PhoneSense Work-shop, 2011.

[40]

Hong Lu, A. J. Bernheim Brush, Bodhi Priyantha, Amy K. Karlson, and Jie Liu. SpeakerSense: energy efficient unobtrusive speaker identification on mobile phones. In Proc. Int'l Conf. on Pervasive Computing, 2011.

Digital Library

Cited By

Alamoudi DNabney ICrawley E(2024)Evaluating the Effectiveness of the SleepTracker App for Detecting Anxiety- and Depression-Related Sleep DisturbancesSensors10.3390/s2403072224:3(722)Online publication date: 23-Jan-2024
https://doi.org/10.3390/s24030722
Kamenjasevic E(2024)Ethical and legal considerations of mood enhancement technologyFrontiers in Human Dynamics10.3389/fhumd.2024.13101446Online publication date: 30-May-2024
https://doi.org/10.3389/fhumd.2024.1310144
Mullick TShaaban SRadovic ADoryab A(2024)Framework for Ranking Machine Learning Predictions of Limited, Multimodal, and Longitudinal Behavioral Passive Sensing Data: Combining User-Agnostic and Personalized ModelingJMIR AI10.2196/478053(e47805)Online publication date: 20-May-2024
https://doi.org/10.2196/47805
Show More Cited By

Index Terms

MoodScope: building a mood sensor from smartphone usage patterns
1. Applied computing
  1. Law, social and behavioral sciences
    1. Psychology
2. Computing methodologies
  1. Machine learning

Recommendations

MoodScope: building a mood sensor from smartphone usage patterns
MobiSys '13: Proceeding of the 11th annual international conference on Mobile systems, applications, and services

We present MoodScope, a software system which infers the mood of its user based on how the smartphone is used. Similar to smartphone sensors that measure acceleration, light, and other physical properties, MoodScope is a "sensor" that measures the ...
Comparing ESM Timings for Emotional Estimation Model with Fine Temporal Granularity
UbiComp '18: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers

Towards wellbeing-awareness in computing, researches for estimating users' emotions using smartphone sensor data have been actively conducted as smartphones are getting more and more ubiquitous. Most studies have constructed emotion estimation models ...
A Large-Scale Study of iPhone App Launch Behaviour
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

There have been many large-scale investigations of users' mobile app launch behaviour, but all have been conducted on Android, even though recent reports suggest iPhones account for a third of all smartphones in use. We report on the first large-scale ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MobiSys '13: Proceeding of the 11th annual international conference on Mobile systems, applications, and services

June 2013

568 pages

ISBN:9781450316729

DOI:10.1145/2462456

General Chairs:
Hao-Hua Chu
National Taiwan University
,
Polly Huang
National Taiwan University
,
Program Chairs:
Romit Roy Choudhury
Duke University
,
Feng Zhao
Microsoft Research

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

In-Cooperation

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MobiSys'13

Sponsor:

SIGMOBILE

MobiSys'13: The 11th Annual International Conference on Mobile Systems, Applications, and Services

June 25 - 28, 2013

Taipei, Taiwan

Acceptance Rates

MobiSys '13 Paper Acceptance Rate 33 of 211 submissions, 16%;

Overall Acceptance Rate 274 of 1,679 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

233
Total Citations
View Citations
3,577
Total Downloads

Downloads (Last 12 months)218
Downloads (Last 6 weeks)19

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Alamoudi DNabney ICrawley E(2024)Evaluating the Effectiveness of the SleepTracker App for Detecting Anxiety- and Depression-Related Sleep DisturbancesSensors10.3390/s2403072224:3(722)Online publication date: 23-Jan-2024
https://doi.org/10.3390/s24030722
Kamenjasevic E(2024)Ethical and legal considerations of mood enhancement technologyFrontiers in Human Dynamics10.3389/fhumd.2024.13101446Online publication date: 30-May-2024
https://doi.org/10.3389/fhumd.2024.1310144
Mullick TShaaban SRadovic ADoryab A(2024)Framework for Ranking Machine Learning Predictions of Limited, Multimodal, and Longitudinal Behavioral Passive Sensing Data: Combining User-Agnostic and Personalized ModelingJMIR AI10.2196/478053(e47805)Online publication date: 20-May-2024
https://doi.org/10.2196/47805
Nepal SLiu WPillai AWang WVojdanovski VHuckins JRogers CMeyer MCampbell A(2024)Capturing the College ExperienceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435018:1(1-37)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643501
Ghosh SMandi SSen SMitra BDe P(2024)Towards Estimating Missing Emotion Self-reports Leveraging User Similarity: A Multi-task Learning ApproachProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642833(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642833
Sakel SBlenk TSchmidt AHaliburton L(2024)The Social Journal: Investigating Technology to Support and Reflect on Social InteractionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642411(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642411
Islam RZhang TBisen PBae S(2024)MoodPupilar: Predicting Mood Through Smartphone Detected Pupillary Responses in Naturalistic Settings2024 IEEE 20th International Conference on Body Sensor Networks (BSN)10.1109/BSN63547.2024.10780658(1-4)Online publication date: 15-Oct-2024
https://doi.org/10.1109/BSN63547.2024.10780658
Sasaki WHamanaka SMiyahara STsubouchi KNakazawa JOkoshi T(2024)Large-Scale Estimation and Analysis of Web Users' Mood from Web Search Query and Mobile Sensor DataBig Data10.1089/big.2022.021112:3(191-209)Online publication date: 1-Jun-2024
https://doi.org/10.1089/big.2022.0211
Farzand HAbraham MBrewster SKhamis MMarky K(2024)A Systematic Deconstruction of Human-Centric Privacy & Security Threats on Mobile PhonesInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2361519(1-24)Online publication date: 12-Jun-2024
https://doi.org/10.1080/10447318.2024.2361519
Balliu BDouglas CSeok DShenhav LWu YChatzopoulou DKaiser WChen VKim JDeverasetty SArnaudova IGibbons RCongdon ECraske MFreimer NHalperin ESankararaman SFlint J(2024)Personalized mood prediction from patterns of behavior collected with smartphonesnpj Digital Medicine10.1038/s41746-024-01035-67:1Online publication date: 28-Feb-2024
https://doi.org/10.1038/s41746-024-01035-6
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents