No-regret Learning Dynamics for Sequential Correlated Equilibria
Author: 
Hugh ZhangAuthors Info & Claims
Hugh ZhangAuthors Info & ClaimsPages 2700 - 2702
Abstract
While no-regret learning procedures that converge to correlated equilibria have long been known to exist for normal form games, their analogue in sequential games remains less clear. We propose the sequential correlated equilibrium, a solution concept that extends the correlated equilibrium to sequential games while also guaranteeing sequential rationality even for mediator recommendations off the path of play. Additionally, we show that any internal regret minimization procedure designed for normal-form games can be efficiently extended to sequential games and use this to design no-regret learning dynamics that converge to the set of sequential correlated equilibria.
References
[1]
Robert J Aumann. 1974. Subjectivity and correlation in randomized strategies. Journal of mathematical Economics, Vol. 1, 1 (1974), 67--96. Publisher: Elsevier.
[2]
David Blackwell. 1956. An analog of the minimax theorem for vector payoffs. Pacific J. Math., Vol. 6, 1 (1956), 1--8. Publisher: Pacific Journal of Mathematics.
[3]
Avrim Blum and Yishay Mansour. 2007. From external to internal regret. Journal of Machine Learning Research, Vol. 8, Jun (2007), 1307--1324.
[4]
Michael Bowling, Neil Burch, Michael Johanson, and Oskari Tammelin. 2015. Heads-up limit hold'em poker is solved. Science, Vol. 347, 6218 (2015), 145--149. Publisher: American Association for the Advancement of Science.
[5]
Noam Brown and Tuomas Sandholm. 2018. Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science, Vol. 359, 6374 (2018), 418--424. Publisher: American Association for the Advancement of Science.
[6]
Noam Brown and Tuomas Sandholm. 2019. Superhuman AI for multiplayer poker. Science, Vol. 365, 6456 (2019), 885--890. Publisher: American Association for the Advancement of Science.
[7]
Andrea Celli, Alberto Marchesi, Gabriele Farina, and Nicola Gatti. 2020. No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium. Advances in Neural Information Processing Systems, Vol. 33 (2020), 7722--7732. https://proceedings.neurips.cc/paper/2020/hash/5763abe87ed1938799203fb6e8650025-Abstract.html
[8]
Xi Chen, Xiaotie Deng, and Shang-Hua Teng. 2009. Settling the complexity of computing two-player Nash equilibria. Journal of the ACM (JACM), Vol. 56, 3 (2009), 1--57. Publisher: ACM New York, NY, USA.
[9]
Constantinos Daskalakis, Paul W. Goldberg, and Christos H. Papadimitriou. 2009. The Complexity of Computing a Nash Equilibrium. SIAM J. Comput., Vol. 39, 1 (Jan. 2009), 195--259. https://doi.org/10.1137/070699652 Publisher: Society for Industrial and Applied Mathematics.
[10]
Francoise Forges. 1986. Correlated equilibria in repeated games with lack of information on one side: a model with verifiable types. International Journal of Game Theory, Vol. 15, 2 (1986), 65--82. Publisher: Springer.
[11]
Dean P Foster and Rakesh V Vohra. 1997. Calibrated learning and correlated equilibrium. Games and Economic Behavior, Vol. 21, 1--2 (1997), 40.
[12]
Sergiu Hart and Andreu Mas-Colell. 2000. A simple adaptive procedure leading to correlated equilibrium. Econometrica, Vol. 68, 5 (2000), 1127--1150. Publisher: Wiley Online Library.
[13]
Sergiu Hart and David Schmeidler. 1989. Existence of Correlated Equilibria. Mathematics of Operations Research, Vol. 14, 1 (1989), 18--25. http://www.jstor.org/stable/3689835 Publisher: INFORMS.
[14]
Wan Huang and Bernhard von Stengel. 2008. Computing an extensive-form correlated equilibrium in polynomial time. In International Workshop on Internet and Network Economics. Springer, 506--513.
[15]
David M Kreps and Robert Wilson. 1982. Sequential equilibria. Econometrica, Vol. 50, 4 (1982), 863--894.
[16]
Matej Moravčík, Martin Schmid, Neil Burch, Viliam Lisý, Dustin Morrill, Nolan Bard, Trevor Davis, Kevin Waugh, Michael Johanson, and Michael Bowling. 2017. DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker. Science, Vol. 356, 6337 (May 2017), 508--513. https://doi.org/10.1126/science.aam6960 arXiv: 1701.01724.
[17]
John Nash. 1951. Non-cooperative games. Annals of mathematics, Vol. 54, 2 (1951), 286--295.
[18]
Aviad Rubinstein. 2019. Hardness of Approximation Between P and NP. Morgan & Claypool.
[19]
Bernhard von Stengel and Francoise Forges. 2008. Extensive-form correlated equilibrium: Definition and computational complexity. Mathematics of Operations Research, Vol. 33, 4 (2008), 1002--1022. Publisher: INFORMS.
[20]
Martin Zinkevich, Michael Johanson, Michael Bowling, and Carmelo Piccione. 2008. Regret minimization in games with incomplete information. In Advances in neural information processing systems. 1729--1736.
Index Terms
- No-regret Learning Dynamics for Sequential Correlated Equilibria
Recommendations
Sequential Stackelberg equilibria in two-person games
The concept of sequential Stackelberg equilibrium is introduced in the general framework of dynamic, two-person games defined in the Denardo contracting operator formalism. A relationship between this solution concept and the sequential Nash equilibrium ...
Computing correlated equilibria in multi-player games
We develop polynomial-time algorithms for finding correlated equilibria—a well-studied notion of rationality that generalizes the Nash equilibrium—in a broad class of succinctly representable multiplayer games, encompassing graphical games, anonymous ...
Using counterfactual regret minimization to create competitive multiplayer poker agents
AAMAS '10: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1Games are used to evaluate and advance Multiagent and Artificial Intelligence techniques. Most of these games are deterministic with perfect information (e.g. Chess and Checkers). A deterministic game has no chance element and in a perfect information ...
Comments
Information & Contributors
Information
Published In
May 2023
3131 pages
ISBN:9781450394321
- General Chairs:
- Noa Agmon,
- Bo An,
- Program Chairs:
- Alessandro Ricci,
- William Yeoh
Sponsors
Publisher
International Foundation for Autonomous Agents and Multiagent Systems
Richland, SC
Publication History
Published: 30 May 2023
Check for updates
Author Tags
Qualifiers
- Poster
Conference
AAMAS '23
Sponsor:
AAMAS '23: International Conference on Autonomous Agents and Multiagent Systems
May 29 - June 2, 2023
London, United Kingdom
Acceptance Rates
Overall Acceptance Rate 1,155 of 5,036 submissions, 23%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 33Total Downloads
- Downloads (Last 12 months)6
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in