skip to main content
research-article

USMART: An Unsupervised Semantic Mining Activity Recognition Technique

Published: 13 November 2014 Publication History

Abstract

Recognising high-level human activities from low-level sensor data is a crucial driver for pervasive systems that wish to provide seamless and distraction-free support for users engaged in normal activities. Research in this area has grown alongside advances in sensing and communications, and experiments have yielded sensor traces coupled with ground truth annotations about the underlying environmental conditions and user actions. Traditional machine learning has had some success in recognising human activities; but the need for large volumes of annotated data and the danger of overfitting to specific conditions represent challenges in connection with the building of models applicable to a wide range of users, activities, and environments. We present USMART, a novel unsupervised technique that combines data- and knowledge-driven techniques. USMART uses a general ontology model to represent domain knowledge that can be reused across different environments and users, and we augment a range of learning techniques with ontological semantics to facilitate the unsupervised discovery of patterns in how each user performs daily activities. We evaluate our approach against four real-world third-party datasets featuring different user populations and sensor configurations, and we find that USMART achieves up to 97.5% accuracy in recognising daily activities.

References

[1]
Ling Bao and Stephen S. Intille. 2004. Activity recognition from user-annotated acceleration data. In Pervasive Computing. Lecture Notes in Computer Science, Vol. 3001. Springer, 1--17.
[2]
Oliver Brdiczka, James L. Crowley, and Patrick Reignier. 2007. Learning situation models for providing context-aware services. In Proceedings of the 4th International Conference on Universal Access in Human-Computer Interaction: Ambient Interaction (UAHCI’07). Springer-Verlag, Berlin, Heidelberg, 23--32. http://dl.acm.org/citation.cfm?id=1763296.1763300
[3]
Hung H. Bui, Svetha Venkatesh, and Geoff West. 2002. Tracking and surveillance in wide-area spatial environments using the abstract hidden Markov model. In Hidden Markov Models: Applications in Computer Vision. World Scientific Publishing, River Edge, NJ, 177--196.
[4]
Liming Chen, Jesse Hoey, Chris D. Nugent, Diane J. Cook, and Zhiwen Yu. 2012a. Sensor-based activity recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 42, 6, 790--808.
[5]
Liming Chen, Chris D. Nugent, and Hui Wang. 2012b. A knowledge-driven approach to activity recognition in smart homes. IEEE Transactions on Knowledge and Data Engineering 24, 961--974.
[6]
Diane Cook. 2012. Learning setting-generalized activity models for smart spaces. IEEE Intelligent Systems 27, 1, 32--38.
[7]
Shai Fine, Yoram Singer, and Naftali Tishby. 1998. The hierarchical hidden Markov model: Analysis and applications. Machine Learning 32, 1, 41--62.
[8]
Tao Gu, Shaxun Chen, Xianping Tao, and Jian Lu. 2010. An unsupervised approach to activity recognition and segmentation based on object-use fingerprints. Data and Knowledge Engineering 69, 6, 533--544.
[9]
Tao Gu, Hung Keng Pung, and Da Qing Zhang. 2005. A service-oriented middleware for building context-aware services. Journal of Network and Computer Applications 28, 1, 1--18.
[10]
Tao Gu, Liang Wang, Zhanqing Wu, Xianping Tao, and Jian Lu. 2011. A pattern mining approach to sensor-based human activity recognition. IEEE Transactions on Knowledge and Data Engineering 23, 9, 1359--1372.
[11]
Tao Gu, Xiao Hang Wang, Hung Keng Pung, and Da Qing Zhang. 2004. An ontology-based context model in intelligent environments. In Proceedings of the International Conference on Communication Network and Distributed Systems (CNDS’04). 270--275.
[12]
Tao Gu, Zhanqing Wu, Xianping Tao, Hung Keng Pung, and Jian Lu. 2009. epSICAR: An emerging patterns based approach to sequential, interleaved and concurrent activity recognition. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom’09). 1--9.
[13]
Jiawei Han, Micheline Kamber, and Jian Pei. 2011. Data Mining: Concepts and Techniques (3rd ed.). Morgan Kaufmann.
[14]
Xin Hong and Chris D. Nugent. 2009. Partitioning time series sensor data for activity recognition. In Proceedings of the 9th International Conference on Information Technology and Applications in Biomedicine (ITAB’09). 1--4.
[15]
Clare J. Hooper, Anne Preston, Madeline Balaam, Paul Seedhouse, Daniel Jackson, Cuong Pham, Cassim Ladha, Karim Ladha, Thomas Plötz, and Patrick Olivier. 2012. The French kitchen: Task-based learning in an instrumented kitchen. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing (UbiComp’12). ACM, New York, NY, 193--202.
[16]
Tâm Huỳnh, Ulf Blanke, and Bernt Schiele. 2007. Scalable recognition of daily activities with wearable sensors. In Proceedings of the 3rd International Conference on Location and Context Awareness (LoCA’07). 50--67. http://dl.acm.org/citation.cfm?id=1777235.1777241
[17]
Jay J. Jiang and David W. Conrath. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of the International Conference Research on Computational Linguistics. 9008+. http://adsabs.harvard.edu/cgi-bin/nph-bib_query?bibcode=1997cmp.lg....9008J
[18]
Tim L. M. Kasteren, Gwenn Englebienne, and Ben J. A. Kröse. 2011. Human activity recognition from wireless sensor network data: Benchmark and software. In Activity Recognition in Pervasive Intelligent Environments. Atlantis Ambient and Pervasive Intelligence, Vol. 4. Springer, 165--186.
[19]
Narayanan C. Krishnan and Diane J. Cook. 2014. Activity recognition on streaming sensor data. Pervasive and Mobile Computing 10, 138--154.
[20]
Javier Ortiz Laguna, Angel García Olaya, and Daniel Borrajo. 2011. A dynamic sliding window approach for activity recognition. In Proceedings of the 19th International Conference on User Modeling, Adaption, and Personalization (UMAP’11). Springer-Verlag, Berlin, Heidelberg, 219--230. http://dl.acm.org/citation.cfm?id=2021855.2021875
[21]
Claudia Leacock, George A. Miller, and Martin Chodorow. 1998. Using corpus statistics and WordNet relations for sense identification. Computational Linguistics 24, 1, 147--165. http://dl.acm.org/citation.cfm?id=972719.972726
[22]
Beth Logan, Jennifer Healey, Matthai Philipose, Emmanuel Munguia Tapia, and Stephen Intille. 2007. A long-term evaluation of sensing modalities for activity recognition. In Proceedings of the 9th International Conference on Ubiquitous Computing (UbiComp’07). Springer-Verlag, Berlin, Heidelberg, 483--500. http://dl.acm.org/citation.cfm?id=1771592.1771620
[23]
Susan McKeever, Juan Ye, Lorcan Coyle, and Simon Dobson. 2010. Activity reasoning using temporal evidence theory. Ambient Intelligence and Smart Environment 2, 3, 253--269.
[24]
George A. Miller. 1995. WordNet: A lexical database for English. Communications of the ACM 38, 11, 39--41.
[25]
George Okeyo, Liming Chen, Hui Wang, and Roy Sterritt. 2011. Ontology-based learning framework for activity assistance in an adaptive smart home. In Activity Recognition in Pervasive Intelligent Environments. Atlantis Ambient and Pervasive Intelligence, Vol. 4. Springer, 237--263.
[26]
George Okeyo, Liming Chen, Hui Wang, and Roy Sterritt. 2014. Dynamic sensor data segmentation for real-time knowledge-driven activity recognition. Pervasive and Mobile Computing 10, 155--172.
[27]
Paulito Palmes, Hung Keng Pung, Tao Gu, Wenwei Xue, and Shaxun Chen. 2010. Object relevance weight pattern mining for activity recognition and segmentation. Pervasive and Mobile Computing 6, 1, 43--57.
[28]
Donald J. Patterson, Dieter Fox, Henry Kautz, and Matthai Philipose. 2005. Fine-grained activity recognition by aggregating abstract object usage. In Proceedings of the 9th IEEE International Symposium on Wearable Computers (ISWC’05). IEEE, Los Alamitos, CA, 44--51.
[29]
Martin Petzold, Francesco Barbabella, Jan Bobeth, Dagmar Kern, Christopher Mayer, and Martin Morandell. 2013. Towards an ambient assisted living user interaction taxonomy. In CHI’13 Extended Abstracts on Human Factors in Computing Systems (CHI EA’13). ACM, New York, NY, 49--54.
[30]
Parisa Rashidi and Diane J. Cook. 2010. Mining sensor streams for discovering human activity patterns over time. In Proceedings of the 10th IEEE International Conference on Data Mining (ICDM’10). IEEE, Los Alamitos, CA, 431--440.
[31]
Parisa Rashidi, Diane J. Cook, Lawrence B. Holder, and Maureen Schmitter-Edgecombe. 2011. Discovering activities to recognize and track in a smart environment. IEEE Transactions on Knowledge and Data Engineering 23, 4, 527--539.
[32]
Philip Resnik. 1995. Using information content to evaluate semantic similarity in a taxonomy. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, Vol. 1. Morgan Kaufmann, San Francisco, CA, 448--453.
[33]
Daniele Riboni and Claudio Bettini. 2009. Context-aware activity recognition through a combination of ontological and statistical reasoning. In Proceedings of the 6th International Conference on Ubiquitous Intelligence and Computing (UIC’09). Springer-Verlag, Berlin, Heidelberg, 39--53.
[34]
Daniel Roggen, Alberto Calatroni, Mirco Rossi, Thomas Holleczek, Killian Forster, Gerhard Troster, Paul Lukowicz, David Bannach, Gerald Pirkl, Alois Ferscha, Jakob Doppler, Clemens Holzmann, Marc Kurz, Gerald Holl, Ricardo Chavarriaga, Hesam Sagha, Hamidreza Bayati, Marco Creatura, and Jose del R. Millan. 2010. Collecting complex activity datasets in highly rich networked sensor environments. In Proceedings of the 7th International Conference on Networked Sensing Systems (INSS’10). 233--240.
[35]
Geetika Singla and Diane J. Cook. 2010. Recognizing independent and joint activities among multiple residents in smart environments. Ambient Intelligent and Humanized Computing 1, 1, 57--83.
[36]
Emmanuel Munguia Tapia, Tanzeem Choudhury, and Matthai Philipose. 2006. Building reliable activity models using hierarchical shrinkage and mined ontology. In Pervasive Computing. Lecture Notes in Computer Science, Vol. 2968, Springer, 17--32.
[37]
Emmanuel Munguia Tapia, Stephen S. Intille, and Kent Larson. 2004a. Activity recognition in the home using simple and ubiquitous sensors. In Pervasive Computing. Lecture Notes in Computer Science, Vol. 3001. Springer, 158--175.
[38]
Emmanuel Munguia Tapia, Stephen S. Intille, and Kent Larson. 2004b. Activity recognition in the home using simple and ubiquitous sensors. In Pervasive Computing. Lecture Notes in Computer Science, Vol. 3001. Springer, 158--175.
[39]
Tim Van Kasteren, Athanasios Noulas, Gwenn Englebienne, and Ben Kröse. 2008. Accurate activity recognition in a home setting. In Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp’08). ACM, New York, NY, 1--9.
[40]
Ulrike Von Luxburg. 2010. Clustering stability: An overview. Foundations and Trends in Machine Learning 3, 2, 235--274.
[41]
Shiaokai Wang, William Pentney, Ana-Maria Popescu, Tanzeem Choudhury, and Matthai Philipose. 2007. Common sense based joint training of human activity recognizers. In Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI’07). Morgan Kaufmann, San Francisco, CA, 2237--2242. http://dl.acm.org/citation.cfm?id=1625275.1625636
[42]
Zhibiao Wu and Martha Palmer. 1994. Verbs semantics and lexical selection. In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics (ACL’94). Association for Computational Linguistics, Stroudsburg, PA, 133--138.
[43]
Juan Ye, Simon Dobson, and Susan McKeever. 2012. Situation identification techniques in pervasive computing: A review. Pervasive and Mobile Computing 8, 1, 36--66.
[44]
Shumin Zhai and Victoria Bellotti. 2005. Introduction to sensing-based interaction. ACM Transactions on Computer-Human Interaction 12, 1, 1--2.
[45]
Vincent Wenchen Zheng, Derek Hao Hu, and Qiang Yang. 2009. Cross-domain activity recognition. In Proceedings of the 11th International Conference on Ubiquitous Computing (Ubicomp’09). ACM, New York, NY, 61--70.

Cited By

View all

Index Terms

  1. USMART: An Unsupervised Semantic Mining Activity Recognition Technique

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Interactive Intelligent Systems
    ACM Transactions on Interactive Intelligent Systems  Volume 4, Issue 4
    Special Issue on Activity Recognition for Interaction and Regular Article
    January 2015
    190 pages
    ISSN:2160-6455
    EISSN:2160-6463
    DOI:10.1145/2688469
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 November 2014
    Accepted: 01 June 2014
    Revised: 01 June 2014
    Received: 01 September 2013
    Published in TIIS Volume 4, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Activity recognition
    2. clustering
    3. ontologies
    4. pervasive computing
    5. segmentation
    6. sequential pattern
    7. smart home
    8. string alignment
    9. unsupervised learning

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)30
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 14 Sep 2024

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media