跳到主要导航 跳到搜索 跳到主要内容

Q-learning solution for optimal consensus control of discrete-time multiagent systems using reinforcement learning

  • Tianjin University
  • Southeast University, Nanjing

科研成果: 期刊稿件 › 文章 › 同行评审

73 引用 (Scopus)

摘要

This paper investigates a Q-learning scheme for the optimal consensus control of discrete-time multiagent systems. The Q-learning algorithm is conducted by reinforcement learning (RL) using system data instead of system dynamics information. In the multiagent systems, the agents are interacted with each other and at least one agent can communicate with the leader directly, which is described by an algebraic graph structure. The objective is to make all the agents achieve synchronization with leader and make the performance indices reach Nash equilibrium. On one hand, the solutions of the optimal consensus control for multiagent systems are acquired by solving the coupled Hamilton–Jacobi–Bellman (HJB) equation. However, it is difficult to get analytical solutions directly of the discrete-time HJB equation. On the other hand, accurate mathematical models of most systems in real world are hard to be obtained. To overcome these difficulties, Q-learning algorithm is developed using system data rather than the accurate system model. We formulate performance index and corresponding Bellman equation of each agent i. Then, the Q-function Bellman equation is acquired on the basis of Q-function. Policy iteration is adopted to calculate the optimal control iteratively, and least square (LS) method is employed to motivate the implementation process. Stability analysis of proposed Q-learning algorithm for multiagent systems by policy iteration is given. Two simulation examples are experimented to verify the effectiveness of the proposed scheme.

源语言英语
页(从-至)6946-6967
页数22
期刊Journal of the Franklin Institute
卷356
期13
DOI
出版状态已出版 - 9月 2019
已对外发布是

学术指纹

探究 'Q-learning solution for optimal consensus control of discrete-time multiagent systems using reinforcement learning' 的科研主题。它们共同构成独一无二的学术指纹。

引用此