demonstration

Teaching agents with human feedback: a demonstration of the TAMER framework

Authors:

IUI '13 Companion: Proceedings of the companion publication of the 2013 international conference on Intelligent user interfaces companion

Pages 65 - 66

https://doi.org/10.1145/2451176.2451201

Published: 19 March 2013 Publication History

Get Access

Abstract

Incorporating human interaction into agent learning yields two crucial benefits. First, human knowledge can greatly improve the speed and final result of learning compared to pure trial-and-error approaches like reinforcement learning. And second, human users are empowered to designate "correct" behavior. In this abstract, we present research on a system for learning from human interaction - the TAMER framework - then point to extensions to TAMER, and finally describe a demonstration of these systems.

References

[1]

Isbell, C., Kearns, M., Singh, S., Shelton, C., Stone, P., and Kormann, D. Cobot in LambdaMOO: An Adaptive Social Statistics Agent. Proceedings of The 5th Annual International Conference on Autonomous Agents and Multiagent Systems (AAMAS) (2006).

Digital Library

Google Scholar

[2]

Knox, W., Glass, B., Love, B., Maddox, W., and Stone, P. How humans teach agents: A new experimental perspective. International Journal of Social Robotics, Special Issue on Robot Learning from Demonstration (2012).

Google Scholar

[3]

Knox, W., and Stone, P. Combining manual feedback with subsequent MDP reward signals for reinforcement learning. Proceedings of The 9th Annual International Conference on Autonomous Agents and Multiagent Systems (AAMAS) (2010).

Digital Library

Google Scholar

[4]

Knox, W. B. Learning from Human-Generated Reward. PhD thesis, Department of Computer Science, The University of Texas at Austin, August 2012.

Google Scholar

[5]

Knox, W. B., and Stone, P. Interactively shaping agents via human reinforcement: The TAMER framework. In The 5th International Conference on Knowledge Capture (September 2009).

Digital Library

Google Scholar

[6]

Knox, W. B., and Stone, P. Reinforcement learning from human reward: Discounting in episodic tasks. In 21st IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man) (September 2012).

Crossref

Google Scholar

[7]

Knox, W. B., and Stone, P. Reinforcement learning with human and MDP reward. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS) (June 2012).

Digital Library

Google Scholar

[8]

Knox, W. B., and Stone, P. Learning non-myopically from human-generated reward. In International Conference on Intelligent User Interfaces (IUI) (March 2013).

Digital Library

Google Scholar

[9]

León, A., Morales, E., Altamirano, L., and Ruiz, J. Teaching a robot to perform task through imitation and on-line feedback. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications (2011), 549--556.

Digital Library

Google Scholar

[10]

Pilarski, P., Dawson, M., Degris, T., Fahimi, F., Carey, J., and Sutton, R. Online human training of a myoelectric prosthesis controller via actor-critic reinforcement learning. In IEEE International Conference on Rehabilitation Robotics (ICORR), IEEE (2011), 1--7.

Crossref

Google Scholar

[11]

Suay, H., and Chernova, S. Effect of human guidance and state space size on interactive reinforcement learning. In 20th IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man) (2011), 1--6.

Crossref

Google Scholar

[12]

Tenorio-Gonzalez, A., Morales, E., and Villaseñor-Pineda, L. Dynamic reward shaping: training a robot by voice. Advances in Artificial Intelligence - IBERAMIA (2010), 483--492.

Digital Library

Google Scholar

[13]

Thomaz, A., and Breazeal, C. Teachable robots: Understanding human teaching behavior to build more effective robot learners. Artificial ntelligence 172, 6-7 (2008), 716--737.

Digital Library

Google Scholar

Cited By

View all

Liu KWang DDu WWu DFu Y(2023)Interactive reinforced feature selection with traverse strategyKnowledge and Information Systems10.1007/s10115-022-01812-365:5(1935-1962)Online publication date: 21-Jan-2023
https://doi.org/10.1007/s10115-022-01812-3
Rabby MKarimoddini AKhan MJiang S(2022)A Learning-Based Adjustable Autonomy Framework for Human–Robot CollaborationIEEE Transactions on Industrial Informatics10.1109/TII.2022.314556718:9(6171-6180)Online publication date: Sep-2022
https://doi.org/10.1109/TII.2022.3145567
Fan WLiu KLiu HGe YXiong HFu Y(2021)Interactive Reinforcement Learning for Feature Selection with Decision Tree in the LoopIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.3102120(1-1)Online publication date: 2021
https://doi.org/10.1109/TKDE.2021.3102120
Show More Cited By

Index Terms

Teaching agents with human feedback: a demonstration of the TAMER framework
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Interactively shaping agents via human reinforcement: the TAMER framework
K-CAP '09: Proceedings of the fifth international conference on Knowledge capture

As computational learning agents move into domains that incur real costs (e.g., autonomous driving or financial investment), it will be necessary to learn good policies without numerous high-cost learning trials. One promising approach to reducing ...
Learning non-myopically from human-generated reward
IUI '13: Proceedings of the 2013 international conference on Intelligent user interfaces

Recent research has demonstrated that human-generated reward signals can be effectively used to train agents to perform a range of reinforcement learning tasks. Such tasks are either episodic - i.e., conducted in unconnected episodes of activity that ...
Framing reinforcement learning from human reward

Several studies have demonstrated that reward from a human trainer can be a powerful feedback signal for control-learning algorithms. However, the space of algorithms for learning from such human reward has hitherto not been explored systematically. ...

Comments

Information & Contributors

Information

Published In

IUI '13 Companion: Proceedings of the companion publication of the 2013 international conference on Intelligent user interfaces companion

March 2013

140 pages

ISBN:9781450319669

DOI:10.1145/2451176

General Chair:
Jihie Kim
University of Southern California, USA
,
Program Chairs:
Jeffrey Nichols
IBM Research -- Almaden, USA
,
Pedro Szekely
University of Southern California, USA

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 March 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Demonstration

Conference

IUI '13

Sponsor:

IUI '13: 18th International Conference on Intelligent User Interfaces

March 19 - 22, 2013

California, Santa Monica, USA

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
231
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Liu KWang DDu WWu DFu Y(2023)Interactive reinforced feature selection with traverse strategyKnowledge and Information Systems10.1007/s10115-022-01812-365:5(1935-1962)Online publication date: 21-Jan-2023
https://doi.org/10.1007/s10115-022-01812-3
Rabby MKarimoddini AKhan MJiang S(2022)A Learning-Based Adjustable Autonomy Framework for Human–Robot CollaborationIEEE Transactions on Industrial Informatics10.1109/TII.2022.314556718:9(6171-6180)Online publication date: Sep-2022
https://doi.org/10.1109/TII.2022.3145567
Fan WLiu KLiu HGe YXiong HFu Y(2021)Interactive Reinforcement Learning for Feature Selection with Decision Tree in the LoopIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.3102120(1-1)Online publication date: 2021
https://doi.org/10.1109/TKDE.2021.3102120
Liu KWang PWang DDu WWu DFu Y(2021)Efficient Reinforced Feature Selection via Early Stopping Traverse Strategy2021 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM51629.2021.00051(399-408)Online publication date: Dec-2021
https://doi.org/10.1109/ICDM51629.2021.00051
Fan WLiu KLiu HWang PGe YFu Y(2020)AutoFS: Automated Feature Selection via Diversity-Aware Interactive Reinforcement Learning2020 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM50108.2020.00117(1008-1013)Online publication date: Nov-2020
https://doi.org/10.1109/ICDM50108.2020.00117
Alonso M(2019)Learning User Preferences via Reinforcement Learning with Spatial Interface ValuingUniversal Access in Human-Computer Interaction. Multimodality and Assistive Environments10.1007/978-3-030-23563-5_32(403-418)Online publication date: 4-Jul-2019
https://doi.org/10.1007/978-3-030-23563-5_32
Cui YNiekum S(2018)Active Reward Learning from Critiques2018 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA.2018.8460854(6907-6914)Online publication date: May-2018
https://doi.org/10.1109/ICRA.2018.8460854
Cruz FMagg SWeber CWermter S(2016)Training Agents With Interactive Reinforcement Learning and Contextual AffordancesIEEE Transactions on Cognitive and Developmental Systems10.1109/TCDS.2016.25438398:4(271-284)Online publication date: Dec-2016
https://doi.org/10.1109/TCDS.2016.2543839
Cruz FTwiefel JMagg SWeber CWermter S(2015)Interactive reinforcement learning through speech guidance in a domestic scenario2015 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN.2015.7280477(1-8)Online publication date: Jul-2015
https://doi.org/10.1109/IJCNN.2015.7280477
Cruz FMagg SWeber CWermter S(2014)Improving reinforcement learning with interactive feedback and affordances4th International Conference on Development and Learning and on Epigenetic Robotics10.1109/DEVLRN.2014.6982975(165-170)Online publication date: Oct-2014
https://doi.org/10.1109/DEVLRN.2014.6982975
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Interactively shaping agents via human reinforcement: the TAMER framework

Learning non-myopically from human-generated reward

Framing reinforcement learning from human reward

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations