skip to main content
10.5555/3545946.3598668acmconferencesArticle/Chapter ViewAbstractPublication PagesaamasConference Proceedingsconference-collections
research-article

Learning Structured Communication for Multi-Agent Reinforcement Learning

Published: 30 May 2023 Publication History

Abstract

This paper investigates multi-agent reinforcement learning (MARL) communication mechanisms in large-scale scenarios. We propose a novel framework, Learning Structured Communication (LSC), that leverages a flexible and efficient communication topology. LSC enables adaptive agent grouping to create diverse hierarchical formations over episodes generated through an auxiliary task and a hierarchical routing protocol. We learn a hierarchical graph neural network with the formed topology that facilitates effective message generation and propagation between inter- and intra-group communications. Unlike state-of-the-art communication mechanisms, LSC possesses a detailed and learnable design for hierarchical communication. Numerical experiments on challenging tasks demonstrate that the proposed LSC exhibits high communication efficiency and global cooperation capability.

References

[1]
Jakob Foerster, Ioannis Alexandros Assael, Nando De Freitas, and Shimon Whiteson. 2016. Learning to communicate with deep multi-agent reinforcement learning. In Neural Information Processing Systems. 2137--2145.
[2]
Jiechuan Jiang, Chen Dun, Tiejun Huang, and Zongqing Lu. 2020. Graph Convolutional Reinforcement Learning. In International Conference on Learning Representations.
[3]
Jiechuan Jiang and Zongqing Lu. 2018. Learning attentional communication for multi-agent cooperation. In Neural Information Processing Systems. 7254--7264.
[4]
M Rezaee and M Yaghmaee. 2009. Cluster based routing protocol for mobile ad hoc networks. IEEE International Conference on Computer Communications, 30--36.
[5]
Sainbayar Sukhbaatar, Rob Fergus, et al. 2016. Learning multiagent communication with backpropagation. In Neural Information Processing Systems. 2244--2252.

Index Terms

  1. Learning Structured Communication for Multi-Agent Reinforcement Learning

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      AAMAS '23: Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems
      May 2023
      3131 pages
      ISBN:9781450394321
      • General Chairs:
      • Noa Agmon,
      • Bo An,
      • Program Chairs:
      • Alessandro Ricci,
      • William Yeoh

      Sponsors

      Publisher

      International Foundation for Autonomous Agents and Multiagent Systems

      Richland, SC

      Publication History

      Published: 30 May 2023

      Check for updates

      Author Tags

      1. graph neural networks
      2. hierarchical structure
      3. learning to communicate
      4. multi-agent reinforcement learning

      Qualifiers

      • Research-article

      Funding Sources

      • Postdoctoral Science Foundation of China
      • Shenzhen Science and Technology Program
      • Shenzhen Institute of Artificial Intelligence and Robotics for Society
      • STCSM
      • NSFC
      • Shanghai Trusted Industry Internet Software Collaborative Innovation Center
      • China Academy of LVT

      Conference

      AAMAS '23
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 1,155 of 5,036 submissions, 23%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • 0
        Total Citations
      • 118
        Total Downloads
      • Downloads (Last 12 months)60
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 27 Jan 2025

      Other Metrics

      Citations

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media