survey

Open access

Multioccupant Activity Recognition in Pervasive Smart Home Environments

Authors:

Asma Benmansour,

Abdelhamid Bouchachia,

Mohammed FehamAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 48, Issue 3

Article No.: 34, Pages 1 - 36

https://doi.org/10.1145/2835372

Published: 09 December 2015 Publication History

Abstract

Human activity recognition in ambient intelligent environments like homes, offices, and classrooms has been the center of a lot of research for many years now. The aim is to recognize the sequence of actions by a specific person using sensor readings. Most of the research has been devoted to activity recognition of single occupants in the environment. However, living environments are usually inhabited by more than one person and possibly with pets. Hence, human activity recognition in the context of multioccupancy is more general, but also more challenging. The difficulty comes from mainly two aspects: resident identification, known as data association, and diversity of human activities. The present survey article provides an overview of existing approaches and current practices for activity recognition in multioccupant smart homes. It presents the latest developments and highlights the open issues in this field.

References

[1]

Giovanni Acampora, Diane J. Cook, Parisa Rashidi, and Athanasios V. Vasilakos. 2013. A survey on ambient intelligence in healthcare. Proc. IEEE 101, 12 (Dec. 2013), 2470--2494.

[2]

Hande Alerndar, Halil Ertan, Ozlern D. Incel, and Cem Ersoy. 2013. ARAS human activity datasets in multiple homes with multiple residents. In Proceedings of the 7th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth’13). 232--235.

Digital Library

[3]

Anon. 2014. Centers for Disease Control and Prevention. Retrieved October 13, 2014 from http://www.cdc.gov/.

[4]

Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, New York.

Digital Library

[5]

Ann Bookman, M. Harrington, L. Pass, and E. Reisner. 2007. Family Caregiver Handbook: Finding Elder Care Resources in Massachusetts. (2007). Retrieved October 15, 2014 from http://web.mit. edu/workplacecenter/hndbk/.

[6]

Abdelhamid Bouchachia and Charlie Vanaret. 2014. GT2FC: An online growing interval type-2 self-learning fuzzy classifier. IEEE Trans. Fuzzy Syst. 22, 4 (Aug. 2014), 999--1018.

[7]

Marie Chan, Eric Campo, Daniel Estève, and Jean-Yves Fourniols. 2009. Smart homes—Current features and future perspectives. Maturitas 64, 2 (Oct. 2009), 90--97.

[8]

Rong Chen and Yu Tong. 2014. A two-stage method for solving multi-resident activity recognition in smart environments. Entropy 16, 4 (April 2014), 2184--2203.

[9]

Yi-ting Chiang, J. Y. J. Hsu, Ching-Hu Lu, and Li-Chen Fu. 2010. Interaction models for multiple-resident activity recognition in a smart home. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’10). 3753--3758.

[10]

Diane J. Cook. 2009. Multi-agent smart environments. J. Ambient Intell. Smart Env. 1, 1 (Jan. 2009), 51--55.

Digital Library

[11]

Diane J. Cook, Aaron Crandall, Geetika Singla, and Brian Thomas. 2010. Detection of social interaction in smart spaces. Cybern. Syst. 41, 2 (Feb. 2010), 90--104.

Digital Library

[12]

Erin Y. Cornwell and Linda J. Waite. 2009. Social disconnectedness, perceived isolation, and health among older adults. J. Health Soc. Behav. 50, 1 (March 2009), 31--48.

[13]

Aaron S. Crandall and Diane J. Cook. 2009. Coping with multiple residents in a smart environment. J. Ambient Intell. Smart Env. 1, 4 (Dec. 2009), 323--334.

Digital Library

[14]

Aaron S. Crandall and Diane J. Cook. 2008a. Resident and caregiver: Handling multiple people in a smart care facility. In Proceedings of the AAAI Fall Symposium—AI in Eldercare: New solutions to Old Problems (2008). 39--47.

[15]

Aaron S. Crandall and Diane J. Cook. 2008b. Attributing events to individuals in multi-inhabitant environments. In Proceedings of the IET 4th International Conference on Intelligent Environments (August 2008). 1--8.

[16]

Aaron S. Crandall and Diane J. Cook. 2010. Using a hidden Markov model for resident identification. In Proceedings of th 6th International Conference on Intelligent Environments (IE’10). 74--79.

Digital Library

[17]

Scott Davidoff, Min Kyung Lee, John Zimmerman, and Anind Dey. 2006. Socially-aware requirements for a smart home. In Proceedings of the International Symposium on Intelligent Environments. 41--44.

[18]

Youtian Du, Feng Chen, and Wenli Xu. 2007. Human interaction representation and recognition through motion decomposition. IEEE Signal Process. Lett. 14, 12 (Dec. 2007), 952--955. doi.org/10.1109/LSP.2007.908035

[19]

Youtian Du, Feng Chen, Wenli Xu, and Yongbin Li. 2006. Recognizing interaction activities using dynamic Bayesian network. In Proceedings of the 18th International Conference on Pattern Recognition (ICPR’06). 618--621.

Digital Library

[20]

Nathan Eagle. 2008. Behavioral inference across cultures: Using telephones as a cultural lens. IEEE Intell. Syst. 23, 4 (July 2008), 62--64.

Digital Library

[21]

Laura Fratiglioni, Hui-Xin Wang, Kjerstin Ericsson, Margaret Maytan, and Bengt Winblad. 2000. Influence of social network on occurrence of dementia: A community-based longitudinal study. Lancet 355, 9212 (April 2000), 1315--1319.

[22]

Jane Gross. 2007. A Grass-Roots Effort to Grow Old at Home. N. Y. Times (August 2007).

[23]

Tao Gu, Zhanqing Wu, Xianping Tao, H. K. Pung, and Jian Lu. 2009a. epSICAR: An emerging patterns based approach to sequential, interleaved and concurrent activity recognition. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom’09). 1--9.

Digital Library

[24]

Tao Gu, Zhanqing Wu, Liang Wang, Xianping Tao, and Jian Lu. 2009b. Mining emerging patterns for recognizing activities of multiple users in pervasive computing. In Proceedings of the 6th Annual International Conference on Mobile and Ubiquitous Systems: Networking Services (MobiQuitous’09). 1--10.

[25]

Chien-Chang Hsu and Liang-Zong Wang. 2008. A smart home resource management system for multiple inhabitants by agent conceding negotiation. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC’08). 607--612. 2008.48 11344

[26]

Kuo-Chung Hsu, Yi-Ting Chiang, Gu-Yang Lin, Ching-Hu Lu, Jane Yung-Jen Hsu, and Li-Chen Fu. 2010. Strategies for inference mechanism of conditional random fields for multiple-resident activity recognition in a smart home. In Trends in Applied Intelligent Systems, Lecture Notes in Computer Science. Springer, Berlin, 417--426.

Digital Library

[27]

Derek Hao Hu, Xian-Xing Zhang, Jie Yin, Vincent Wenchen Zheng, and Qiang Yang. 2009. Abnormal activity recognition based on HDP-HMM models. In Proceedings of the 21st International Joint Conference on Artifical Intelligence (IJCAI’09). Morgan Kaufmann Publishers Inc., 1715--1720.

Digital Library

[28]

Shehroz S. Khan, Michelle E. Karg, Jesse Hoey, and Dana Kulic. 2012. Towards the detection of unusual temporal events during activities using HMMs. In Proceedings of the Conference on Ubiquitous Computing (UbiComp’12). ACM, New York, 1075--1084.

Digital Library

[29]

Eunju Kim, S. Helal, and D. Cook. 2010. Human activity recognition and pattern discovery. IEEE Pervasive Comput. 9, 1 (Jan. 2010), 48--53.

Digital Library

[30]

Jinyan Li, Guimei Liu, and Limsoon Wong. 2007. Mining statistically important equivalence classes and delta-discriminative emerging patterns. In Proceedings of the 13th International Conference on Knowledge Discovery and Data Mining (SIGKDD’07). ACM, New York, 430--439.

Digital Library

[31]

Wenhui Liao and Qiang Ji. 2009. Learning Bayesian network parameters under incomplete data with domain knowledge. Pattern Recognit. 42 (2009), 3046--3056.

Digital Library

[32]

Zhi-Hao Lin and Li-Chen Fu. 2007. Multi-user preference model and service provision in a smart home environment. In Proceedings of the IEEE International Conference on Automation Science and Engineering (CASE’07). 759--764.

[33]

Ching-Hu Lu, Yung-Ching Lin, and Li-Chen Fu. 2008. Hide and not easy to seek: A hybrid weaving strategy for context-aware service provision in a smart home. In IEEE Asia-Pacific Services Computing Conference (APSCC’08). 595--600.

Digital Library

[34]

Iain McCowan, Daniel Gatica-Perez, Samy Bengio, Guillaume Lathoud, Mark Barnard, and Dong Zhang. 2005. Automatic analysis of multimodal group actions in meetings. IEEE Trans. Pattern Anal. Mach. Intell. 27, 3 (March 2005), 305--317.

Digital Library

[35]

Ahmed Nait Aicha, Gwenn Englebienne, and Ben Kröse. 2013. How lonely is your grandma?: Detecting the visits to assisted living elderly from wireless sensor network data. In Proceedings of the Conference on Pervasive and Ubiquitous Computing Adjunct Publication (UbiComp’13). ACM, New York, 1285--1294.

Digital Library

[36]

Pradeep Natarajan and Ramakant Nevatia. 2007. Coupled hidden semi Markov models for activity recognition. In Proceedings of the Workshop on Motion and Video Computing (WMVC’07). IEEE, 10.

Digital Library

[37]

Nam Nguyen, Svetha Venkatesh, and Hung Bui. 2006. Recognising behaviours of multiple people with hierarchical probabilistic model and statistical data association. In Proceedings of the 17th British Machine Vision Conference (BMVC’06). The British Machine Vision Association and Society for Pattern Recognition, 1239--1248.

[38]

Daniel O. Olguin, Benjamin N. Waber, Taemie Kim, Akshay Mohan, Koji Ara, and Alex Pentland. 2009. Sensible organizations: Technology and methodology for automatically measuring organizational behavior. IEEE Trans. Syst. Man Cybern. Part B Cybern. 39, 1 (Feb. 2009), 43--55.

Digital Library

[39]

Markus Prossegger and Abdelhamid Bouchachia. 2014. Multi-resident activity recognition using incremental decision trees. In Adaptive and Intelligent Systems, Abdelhamid Bouchachia (Ed.). Lecture Notes in Computer Science. Springer International Publishing, 182--191.

[40]

Iulian Pruteanu-Malinici and Lawrence Carin. 2008. Infinite hidden Markov models for unusual-event detection in video. IEEE Trans. Image Process. 17, 5 (May 2008), 811--822. org/10.1109/TIP.2008.919359

Digital Library

[41]

Barry Reisberg, Sanford I. Finkel, John Overall, and Hellmut Erzigkeit. 2001. The Alzheimer's disease activities of daily living international scale (ADL-IS). Int. Psychogeriatr. IPA 13, 2 (June 2001), 163--181.

[42]

Daniele Riboni, Linda Pareschi, Laura Radaelli, and Claudio Bettini. 2011. Is ontology-based activity recognition really effective? In Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops’11). 427--431.

[43]

Susanne Smith Roley et al. 2008. Occupational therapy practice framework: Domain and practice 2nd ed. Am. J. Occup. Ther. Off. Publ. Am. Occup. Ther. Assoc. 62, 6 (Dec. 2008), 625--683.

[44]

Fariba Sadri. 2011. Ambient intelligence: A survey. ACM Comput. Surv. 43, 4 (Oct. 2011), 36:1--36:66.

Digital Library

[45]

Sumit Sanghai, Pedro Domingos, and Daniel Weld. 2005. Relational dynamic Bayesian networks. J. Artif. Intell. Res. 24, 2005 (2005), 759--797.

[46]

Jehad A. M. Sarkar, Young-Koo Lee, and Sungyoung Lee. 2010. ARHMAM: An activity recognition system based on hidden Markov minded activity model. In Proceedings of the 4th International Conference on Uniquitous Information Management and Communication (ICUIMC’10). ACM, New York, 71:1--71:9.

Digital Library

[47]

Geetika Singla, Diane J. Cook, and Maureen Schmitter-Edgecombe. 2010. Recognizing independent and joint activities among multiple residents in smart environments. J. Ambient Intell. Humaniz. Comput. 1, 1 (March 2010), 57--63.

[48]

Eliot R. Smith and Diane M. Mackie. 1999. Social Psychology. Routledge, London.

[49]

Charles Sutton and Andrew McCallum. 2006. An introduction to conditional random fields for relational learning. In Introduction to Statistical Relational Learning. MIT Press, 93--128.

[50]

Charles Sutton, Andrew McCallum, and Khashayar Rohanimanesh. 2007. Dynamic conditional random fields: Factorized probabilistic models for labeling and segmenting sequence data. J. Mach. Learn. Res. 8 (May 2007), 693--723.

Digital Library

[51]

Tanzeem Choudhury and Alex Pentland. 2004. Characterizing social networks using the sociometer. In Proceedings of NAACOS 2004.

[52]

Tim L. M. van Kasteren. 2011. Activity Recognition for Health Monitoring Elderly Using Temporal Probabilistic Models. Ph.D. dissertation. University of Amsterdam.

[53]

Tim L. M. van Kasteren, Gwenn Englebienne, and Ben J. A. Kröse. 2011. Hierarchical activity recognition using automatically clustered actions. In Proceedings of the 2nd International Conference on Ambient Intelligence (AmI’11). 82--91.

Digital Library

[54]

Tim L. M. van Kasteren, Gwenn Englebienne, and Ben J. A. Kröse. 2010. Human activity recognition from wireless sensor network data: Benchmark and software. In Activity Recognition in Pervasive Intelligent Environments, Liming Chen, Chris D. Nugent, Jit Biswas, and Jesse Hoey (Eds.), Atlantis Ambient and Pervasive Intelligence. Atlantis Press, 165--186.

[55]

Tim L. M. van Kasteren, Athanasios Noulas, Gwenn Englebienne, and Ben Kröse. 2008. Accurate activity recognition in a home setting. In Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp’08). ACM, 1--9.

Digital Library

[56]

Kristof Van Laerhoven, Nicky Kern, Hans W. Gellersen, and Bernt Schiele. 2003. Towards a wearable inertial sensor network. In Proceedings of the IEE Eurowearable Conference (2003). 125--130.

[57]

Liang Wang, Tao Gu, Xianping Tao, Hanhua Chen, and Jian Lu. 2011. Recognizing multi-user activities using wearable sensors in a smart home. Pervasive Mob. Comput. 7, 3 (June 2011), 287--298.

Digital Library

[58]

Liang Wang, Tao Gu, Xianping Tao, and Jian Lu. 2009. Sensor-based human activity recognition in a multi-user scenario. In Ambient Intelligence, Manfred Tscheligi et al. (Eds.). Lecture Notes in Computer Science. Springer, Berlin, 78--87.

Digital Library

[59]

Daniel H. Wilson and Chris Atkeson. 2005. Simultaneous tracking and activity recognition (STAR) using many anonymous, binary sensors. In Pervasive Computing, Hans-W. Gellersen, Roy Want, and Albrecht Schmidt (Eds.). Lecture Notes in Computer Science. Springer, Berlin, 62--79.

Digital Library

Cited By

Arrotta LCivitarese GChen XCumin JBettini C(2025)Multi-subject human activities: A survey of recognition and evaluation methods based on a formal frameworkExpert Systems with Applications10.1016/j.eswa.2024.126178267(126178)Online publication date: Apr-2025
https://doi.org/10.1016/j.eswa.2024.126178
Hao HZhao SChen GZhao SGe Y(2024)Segmentation of multi-residential activities based on spatial correlation and FWC-CPD2024 International Conference on Networking and Network Applications (NaNA)10.1109/NaNA63151.2024.00080(444-452)Online publication date: 9-Aug-2024
https://doi.org/10.1109/NaNA63151.2024.00080
Kim HSon HLee D(2024)Causality-Aware Pattern Mining Scheme for Group Activity Recognition2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC61105.2024.00126(908-917)Online publication date: 2-Jul-2024
https://doi.org/10.1109/COMPSAC61105.2024.00126
Show More Cited By

Index Terms

Multioccupant Activity Recognition in Pervasive Smart Home Environments
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
2. Theory of computation
  1. Logic

Recommendations

Analytical Review on Ontological Human Activity Recognition Approaches

New advancements in pervasive computing technology have turned smart homes into a daily living monitoring tool increasingly used for elderly. Recently, using knowledge driven approaches such as ontology to introduce semantic smart homes has received ...
Annotating smart environment sensor data for activity learning
Smart Environments: Technology to Support Healthcare

The pervasive sensing technologies found in smart homes offer unprecedented opportunities for providing health monitoring and assistance to individuals experiencing difficulties living independently at home. In order to monitor the functional health of ...
Human Activity Behavioural Pattern Recognition in Smart Home with Long-Hour Data Collection
Abstract
The research on human activity recognition has provided novel solutions to many applications like health care, sports, and user profiling. Considering the complex nature of human activities, it is still challenging even after effective and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 48, Issue 3

February 2016

619 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/2856149

Editor:
Sartaj Sahni
Department of Computer and Information Science and Engineering/University of Florida/Gainesville

Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 December 2015

Accepted: 01 September 2015

Revised: 01 June 2015

Received: 01 February 2015

Published in CSUR Volume 48, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Survey
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

102
Total Citations
View Citations
3,295
Total Downloads

Downloads (Last 12 months)241
Downloads (Last 6 weeks)35

Reflects downloads up to 24 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Arrotta LCivitarese GChen XCumin JBettini C(2025)Multi-subject human activities: A survey of recognition and evaluation methods based on a formal frameworkExpert Systems with Applications10.1016/j.eswa.2024.126178267(126178)Online publication date: Apr-2025
https://doi.org/10.1016/j.eswa.2024.126178
Hao HZhao SChen GZhao SGe Y(2024)Segmentation of multi-residential activities based on spatial correlation and FWC-CPD2024 International Conference on Networking and Network Applications (NaNA)10.1109/NaNA63151.2024.00080(444-452)Online publication date: 9-Aug-2024
https://doi.org/10.1109/NaNA63151.2024.00080
Kim HSon HLee D(2024)Causality-Aware Pattern Mining Scheme for Group Activity Recognition2024 IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC)10.1109/COMPSAC61105.2024.00126(908-917)Online publication date: 2-Jul-2024
https://doi.org/10.1109/COMPSAC61105.2024.00126
Singh DMerdivan EKropf JHolzinger A(2024)Class imbalance in multi-resident activity recognition: an evaluative study on explainability of deep learning approachesUniversal Access in the Information Society10.1007/s10209-024-01123-0Online publication date: 13-Jun-2024
https://doi.org/10.1007/s10209-024-01123-0
Ghosh MChoudhury S(2024)A Logical Model for Multiple People Activity Recognition Using Non-intrusive Sensors for Geriatric CareAdvances in Distributed Computing and Machine Learning10.1007/978-981-97-3523-5_8(99-110)Online publication date: 3-Aug-2024
https://doi.org/10.1007/978-981-97-3523-5_8
Imad MCleland INugent CMcAllister P(2024)Integrating Traditional Machine Learning Approaches with Explainable Anomaly Detection for Multimodal Sensor DataProceedings of the International Conference on Ubiquitous Computing and Ambient Intelligence (UCAmI 2024)10.1007/978-3-031-77571-0_11(99-111)Online publication date: 21-Dec-2024
https://doi.org/10.1007/978-3-031-77571-0_11
Song ZChaki DLakhdari ABouguettaya A(2023)Positional Encoding-based Resident Identification in Multi-resident Smart HomesACM Transactions on Internet Technology10.1145/363135324:1(1-27)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1145/3631353
Huang XZhang S(2023)Human Activity Recognition based on Transformer in Smart HomeProceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning10.1145/3590003.3590100(520-525)Online publication date: 17-Mar-2023
https://dl.acm.org/doi/10.1145/3590003.3590100
Lee TKim HLee DHong JLanperne MPark JCerny TShahriar H(2023)Transformer based Early Classification for Real-time Human Activity Recognition in Smart HomesProceedings of the 38th ACM/SIGAPP Symposium on Applied Computing10.1145/3555776.3577693(410-417)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3555776.3577693
Alqahtani EJanbi NSharaf SMehmood R(2022)Smart Homes and Families to Enable Sustainable Societies: A Data-Driven Approach for Multi-Perspective Parameter Discovery Using BERT ModellingSustainability10.3390/su14201353414:20(13534)Online publication date: 19-Oct-2022
https://doi.org/10.3390/su142013534
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents