Article

Adaptive cognitive orthotics: combining reinforcement learning and constraint-based temporal reasoning

Authors:

Matthew Rudary,

Satinder Singh,

Martha E. PollackAuthors Info & Claims

ICML '04: Proceedings of the twenty-first international conference on Machine learning

Page 91

https://doi.org/10.1145/1015330.1015411

Published: 04 July 2004 Publication History

Get Access

Abstract

Reminder systems support people with impaired prospective memory and/or executive function, by providing them with reminders of their functional daily activities. We integrate temporal constraint reasoning with reinforcement learning (RL) to build an adaptive reminder system and in a simulated environment demonstrate that it can personalize to a user and adapt to both short- and long-term changes. In addition to advancing the application domain, our integrated algorithm contributes to research on temporal constraint reasoning by showing how RL can select an optimal policy from amongst a set of temporally consistent ones, and it contributes to the work on RL by showing how temporal constraint reasoning can be used to dramatically reduce the space of actions from which an RL agent needs to learn.

References

[1]

Dechter, R., Meiri, I., & Pearl, J. (1991). Temporal constraint networks. Artificial Intelligence, 49, 61--95.

Digital Library

Google Scholar

[2]

Fiechter, C., & Rogers, S. (2000). Learning subjective functions with large margins. Proceedings of the 17th International Conference on Machine Learning (pp. 287--294).

Digital Library

Google Scholar

[3]

Gervasio, M., Iba, T., & Langley, P. (1999). Learning user evaluation functions for adaptive scheduling assistance. Proceedings of the 16th International Conference on Machine Learning (pp. 152--161).

Digital Library

Google Scholar

[4]

LoPresti, E. F., Mihailidis, A., & Kirsch, N. (2004). Assistive technology for cognitive rehabilitation: State of the art. Neuropsychological Rehabilitation, 14, 5--39.

Crossref

Google Scholar

[5]

Pollack, M. E., Brown, L., Colbry, D., McCarthy, C., Peintner, B., Ramakrishnan, S., & Tsamardinos, I. (2003). Autominder: An intelligent cognitive orthotic system for people with memory impairment. Robotics and Autonomous Systems, 44, 273--282.

Crossref

Google Scholar

[6]

Roy, N., Pineau, J., & Thrun, S. (2000). Spoken dialogue management for robots. Proceedings of the 38th Annual Meeting of the Assn. for Computational Linguistics.

Digital Library

Google Scholar

[7]

Singh, S., Litman, D., Kearns, M., & Walker, M. (2002). Optimizing dialogue management with reinforcement learning: Experiments with the njfun system. Journal of Artificial Intelligence Research, 16, 105--133.

Crossref

Google Scholar

[8]

Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.

Digital Library

Google Scholar

[9]

Thompson, C., Goker, M., & Langley, P. (2004). A personalized system for conversational recommendations. Journal of Artificial Intelligence Research, 21, 393--428.

Crossref

Google Scholar

[10]

Tsamardinos, I., & Pollack, M. E. (2003). Efficient solution techniques for disjunctive temporal problems. Artificial Intelligence, 151, 43--90.

Digital Library

Google Scholar

[11]

Watkins, C. J. C. H. (1989). Learning from delayed rewards. Doctoral dissertation, King's College, Cambridge.

Google Scholar

[12]

Zhang, W., & Dietterich, T. G. (1995). A reinforcement learning approach to job-shop scheduling. Proceedings of the International Joint Conference on Artificial Intellience (pp. 1114--1120).

Digital Library

Google Scholar

Cited By

View all

Zini FLe Piane FGaspari M(2022)Adaptive Cognitive Training with Reinforcement LearningACM Transactions on Interactive Intelligent Systems10.1145/347677712:1(1-29)Online publication date: 4-Mar-2022
https://dl.acm.org/doi/10.1145/3476777
Patel AShah J(2019)Sensor-based activity recognition in the context of ambient assisted living systems: A reviewJournal of Ambient Intelligence and Smart Environments10.3233/AIS-19052911:4(301-322)Online publication date: 23-Jul-2019
https://doi.org/10.3233/AIS-190529
Patel AShah J(2019)Performance Analysis of Supervised Machine Learning Algorithms to Recognize Human Activity in Ambient Assisted Living Environment2019 IEEE 16th India Council International Conference (INDICON)10.1109/INDICON47234.2019.9030353(1-4)Online publication date: Dec-2019
https://doi.org/10.1109/INDICON47234.2019.9030353
Show More Cited By

Adaptive cognitive orthotics: combining reinforcement learning and constraint-based temporal reasoning
1. Computing methodologies

Recommendations

Exploiting Cognitive Structure for Adaptive Learning
KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Adaptive learning, also known as adaptive teaching, relies on learning path recommendation, which sequentially recommends personalized learning items (e.g., lectures, exercises) to satisfy the unique needs of each learner. Although it is well known that ...
Adaptive personalized recommendation based on adaptive learning

Collaborative filtering has been widely applied in many fields in recent years due to the increase in web-based activities such as e-commerce and online content distribution. Current collaborative filtering techniques such as correlation-based, SVD-...
The Adaptive Learning System Based on Learning Style and Cognitive State
KAM '08: Proceedings of the 2008 International Symposium on Knowledge Acquisition and Modeling

Adaptive learning system, in essence, is a kind of online learning environment that supports the individual learning. It changes the traditional "just put it on the web" approach, and provides the customized learning according to the individual ...

Comments

Information & Contributors

Information

Published In

ICML '04: Proceedings of the twenty-first international conference on Machine learning

July 2004

934 pages

ISBN:1581138385

DOI:10.1145/1015330

Conference Chair:
Carla Brodley
Purdue University/Tufts University

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 July 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
531
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zini FLe Piane FGaspari M(2022)Adaptive Cognitive Training with Reinforcement LearningACM Transactions on Interactive Intelligent Systems10.1145/347677712:1(1-29)Online publication date: 4-Mar-2022
https://dl.acm.org/doi/10.1145/3476777
Patel AShah J(2019)Sensor-based activity recognition in the context of ambient assisted living systems: A reviewJournal of Ambient Intelligence and Smart Environments10.3233/AIS-19052911:4(301-322)Online publication date: 23-Jul-2019
https://doi.org/10.3233/AIS-190529
Patel AShah J(2019)Performance Analysis of Supervised Machine Learning Algorithms to Recognize Human Activity in Ambient Assisted Living Environment2019 IEEE 16th India Council International Conference (INDICON)10.1109/INDICON47234.2019.9030353(1-4)Online publication date: Dec-2019
https://doi.org/10.1109/INDICON47234.2019.9030353
Venkatesh JAksanli BChan CAkyurek ARosing T(2018)Modular and Personalized Smart Health Application Design in a Smart City EnvironmentIEEE Internet of Things Journal10.1109/JIOT.2017.27125585:2(614-623)Online publication date: Apr-2018
https://doi.org/10.1109/JIOT.2017.2712558
Nazerfard E(2018)Temporal features and relations discovery of activities from sensor dataJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-018-0855-715:3(1911-1926)Online publication date: 19-May-2018
https://doi.org/10.1007/s12652-018-0855-7
Fortin-Simard DBilodeau JGaboury SBouchard BBouzouane A(2016)Method of recognition and assistance combining passive RFID and electrical load analysis that handles cognitive errorsInternational Journal of Distributed Sensor Networks10.1155/2015/6432732015(190-190)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1155/2015/643273
Das BCook DKrishnan NSchmitter-Edgecombe M(2016)One-Class Classification-Based Real-Time Activity Error Detection in Smart HomesIEEE Journal of Selected Topics in Signal Processing10.1109/JSTSP.2016.253597210:5(914-923)Online publication date: Aug-2016
https://doi.org/10.1109/JSTSP.2016.2535972
Venkatesh JChan CAkyurek ARosing T(2016)A Modular Approach to Context-Aware IoT Applications2016 IEEE First International Conference on Internet-of-Things Design and Implementation (IoTDI)10.1109/IoTDI.2015.13(235-240)Online publication date: Apr-2016
https://doi.org/10.1109/IoTDI.2015.13
Jean-Baptiste ERussell MHowe JRotshtein P(2015)CogWatch: Intelligent agent-based system to assist stroke survivors during tea-making2015 SAI Intelligent Systems Conference (IntelliSys)10.1109/IntelliSys.2015.7361179(452-456)Online publication date: Nov-2015
https://doi.org/10.1109/IntelliSys.2015.7361179
Belley CGaboury SBouchard BBouzouane A(2015)Nonintrusive system for assistance and guidance in smart homes based on electrical devices identificationExpert Systems with Applications: An International Journal10.1016/j.eswa.2015.04.02442:19(6552-6577)Online publication date: 1-Nov-2015
https://dl.acm.org/doi/10.1016/j.eswa.2015.04.024
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Exploiting Cognitive Structure for Adaptive Learning

Adaptive personalized recommendation based on adaptive learning

The Adaptive Learning System Based on Learning Style and Cognitive State