skip to main content
10.1145/2461446.2461449acmconferencesArticle/Chapter ViewAbstractPublication PagescpsweekConference Proceedingsconference-collections
research-article

An inverse correlated equilibrium framework for utility learning in multiplayer, noncooperative settings

Published: 09 April 2013 Publication History

Abstract

In a game-theoretic framework, given parametric agent utility functions, we solve the inverse problem of computing the feasible set of utility function parameters for each individual agent, given that they play a correlated equilibrium strategy. We model agents as utility maximizers, then cast the problem of computing the parameters of players' utility functions as a linear program using the fact that their play results in a correlated equilibrium. We focus on situations where agents must make tradeoffs between multiple competing components within their utility function. We test our method first on a simulated game of Chicken-Dare, and then on data collected in a real-world trial of a mobile fitness game in which five players must balance between protecting their privacy and receiving a reward for burning calories and improving their physical fitness. Through the learned utility functions from the fitness game, we hope to gain insight into the relative importance each user places on safeguarding their privacy vs. achieving the other desirable objectives in the game.

References

[1]
R. Aumann. Correlated equilibrium as an expression of Bayesian rationality. Econometrica: Journal of the Econometric Society, pages 1--18, 1987.
[2]
R. Aumann. Subjectivity and correlation in randomized strategies. Journal of Mathematical Economics, 1(1):67--96, 2012.
[3]
A. Blum and Y. Monsour. Algorithmic Game Theory, Learning, regret minimization, and equilibria. Cambridge University Press, 2007.
[4]
D. Foster and R. Vohra. Regret in the on-line decision problem. Games and Economic Behavior, 29(1):7--35, 1999.
[5]
S. Hart and A. Mas-Colell. Uncoupled dynamics do not lead to Nash equilibrium. The American Economic Review, 93(5):1830--1836, 2003.
[6]
M. Li, R. Poovendran, and S. Narayanan. Protecting patient privacy against unauthorized release of medical images in a group communication environment. Computerized Medical Imaging and Graphics, 29(5):367--383, 2005.
[7]
L. Mac Dermed, C. Isbell, and L. Weiss. Markov games of incomplete information for multi-agent reinforcement learning. In Workshops at the 25th AAAI Conference on Artificial Intelligence, 2011.
[8]
J. Mathe, A. Ledeczi, A. Nadas, J. Sztipanovits, J. Martin, L. Weavind, A. Miller, P. Miller, and D. Maron. A model-integrated, guideline-driven, clinical decision-support system. IEEE Software, 26(4):54--61, july-aug. 2009.
[9]
J. L. Mathe, S. Duncavage, J. Werner, B. A. Malin, A. Ledeczi, and J. Sztipanovits. Towards the security and privacy analysis of patient portals. ACM SIGBED Review, 4(2):5--9, Apr. 2007.
[10]
A. Ng and S. Russell. Algorithms for inverse reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning, pages 663--670, 2000.
[11]
L. Ortiz, R. Schapire, and S. Kakade. Maximum entropy correlated equilibria. Technical report, Massachusetts Institute of Technology Computer Science and Artificial Intelligence Laboratory, 2006.
[12]
C. H. Papadimitriou. Algorithmic Game Theory, The complexity of finding Nash equilibria. Cambridge University Press, 2007.
[13]
C. H. Papadimitriou and T. Roughgarden. Computing correlated equilibria in multi-player games. Journal of the ACM, 55(3):14:1--14:29, Aug. 2008.
[14]
E. A. R. Breu, J. Sztipanovits. Model-based design of trustworthy health information systems. Methods of Information in Medicine, 47(5):389--91, 2008.
[15]
C. Rothkopf and C. Dimitrakakis. Preference elicitation and inverse reinforcement learning. Machine Learning and Knowledge Discovery in Databases, pages 34--48, 2011.
[16]
N. D. Stein, P. A. Parrilo, and A. Ozdaglar. Correlated equilibria in continuous games: Characterization and computation. Games and Economic Behavior, 71(2):436--455, 2011.
[17]
G. Stoltz and G. Lugosi. Learning correlated equilibria in games with compact sets of strategies. Games and Economic Behavior, 59(1):187--208, 2007.
[18]
K. Waugh, B. Ziebart, and J. Bagnell. Computational rationalization: The inverse equilibrium problem. Proceedings of the 28th International Conference on Machine Learning (ICML), 2011.

Cited By

View all

Index Terms

  1. An inverse correlated equilibrium framework for utility learning in multiplayer, noncooperative settings

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    HiCoNS '13: Proceedings of the 2nd ACM international conference on High confidence networked systems
    April 2013
    152 pages
    ISBN:9781450319614
    DOI:10.1145/2461446
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 09 April 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. game theory
    2. mobile health
    3. privacy
    4. utility learning

    Qualifiers

    • Research-article

    Conference

    HiCoNS '13
    Sponsor:

    Acceptance Rates

    HiCoNS '13 Paper Acceptance Rate 18 of 37 submissions, 49%;
    Overall Acceptance Rate 30 of 55 submissions, 55%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)9
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 24 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media