A Data-Based Feedback Relearning Algorithm for Uncertain Nonlinear Systems

Chaoxu Mu; Yong Zhang; Guangbin Cai; Ruijun Liu; Changyin Sun

doi:10.1109/JAS.2023.123186

A Data-Based Feedback Relearning Algorithm for Uncertain Nonlinear Systems

Chaoxu Mu
, Yong Zhang
, Guangbin Cai
, Ruijun Liu
, Changyin Sun

Tianjin University
Rocket Force University of Engineering
Beijing Technology and Business University
Southeast University, Nanjing

Research output: Contribution to journal › Article › peer-review

8 Scopus citations

Abstract

In this paper, a data-based feedback relearning algorithm is proposed for the robust control problem of uncertain nonlinear systems. Motivated by the classical on-policy and off-policy algorithms of reinforcement learning, the online feedback relearning (FR) algorithm is developed where the collected data includes the influence of disturbance signals. The FR algorithm has better adaptability to environmental changes (such as the control channel disturbances) compared with the off-policy algorithm, and has higher computational efficiency and better convergence performance compared with the on-policy algorithm. Data processing based on experience replay technology is used for great data efficiency and convergence stability. Simulation experiments are presented to illustrate convergence stability, optimality and algorithmic performance of FR algorithm by comparison.

Original language	English
Pages (from-to)	1288-1303
Number of pages	16
Journal	IEEE/CAA Journal of Automatica Sinica
Volume	10
Issue number	5
DOIs	https://doi.org/10.1109/JAS.2023.123186
State	Published - 1 May 2023
Externally published	Yes

Keywords

Data episodes
experience replay
neural networks
reinforcement learning (RL)
uncertain systems

Access to Document

10.1109/JAS.2023.123186

Cite this

@article{2bfaf500d06444778dcd58b0e46d06d4,

title = "A Data-Based Feedback Relearning Algorithm for Uncertain Nonlinear Systems",

abstract = "In this paper, a data-based feedback relearning algorithm is proposed for the robust control problem of uncertain nonlinear systems. Motivated by the classical on-policy and off-policy algorithms of reinforcement learning, the online feedback relearning (FR) algorithm is developed where the collected data includes the influence of disturbance signals. The FR algorithm has better adaptability to environmental changes (such as the control channel disturbances) compared with the off-policy algorithm, and has higher computational efficiency and better convergence performance compared with the on-policy algorithm. Data processing based on experience replay technology is used for great data efficiency and convergence stability. Simulation experiments are presented to illustrate convergence stability, optimality and algorithmic performance of FR algorithm by comparison.",

keywords = "Data episodes, experience replay, neural networks, reinforcement learning (RL), uncertain systems",

author = "Chaoxu Mu and Yong Zhang and Guangbin Cai and Ruijun Liu and Changyin Sun",

note = "Publisher Copyright: {\textcopyright} 2014 Chinese Association of Automation.",

year = "2023",

month = may,

day = "1",

doi = "10.1109/JAS.2023.123186",

language = "英语",

volume = "10",

pages = "1288--1303",

journal = "IEEE/CAA Journal of Automatica Sinica",

issn = "2329-9266",

publisher = "IEEE Advancing Technology for Humanity",

number = "5",

}

TY - JOUR

T1 - A Data-Based Feedback Relearning Algorithm for Uncertain Nonlinear Systems

AU - Mu, Chaoxu

AU - Zhang, Yong

AU - Cai, Guangbin

AU - Liu, Ruijun

AU - Sun, Changyin

PY - 2023/5/1

Y1 - 2023/5/1

N2 - In this paper, a data-based feedback relearning algorithm is proposed for the robust control problem of uncertain nonlinear systems. Motivated by the classical on-policy and off-policy algorithms of reinforcement learning, the online feedback relearning (FR) algorithm is developed where the collected data includes the influence of disturbance signals. The FR algorithm has better adaptability to environmental changes (such as the control channel disturbances) compared with the off-policy algorithm, and has higher computational efficiency and better convergence performance compared with the on-policy algorithm. Data processing based on experience replay technology is used for great data efficiency and convergence stability. Simulation experiments are presented to illustrate convergence stability, optimality and algorithmic performance of FR algorithm by comparison.

AB - In this paper, a data-based feedback relearning algorithm is proposed for the robust control problem of uncertain nonlinear systems. Motivated by the classical on-policy and off-policy algorithms of reinforcement learning, the online feedback relearning (FR) algorithm is developed where the collected data includes the influence of disturbance signals. The FR algorithm has better adaptability to environmental changes (such as the control channel disturbances) compared with the off-policy algorithm, and has higher computational efficiency and better convergence performance compared with the on-policy algorithm. Data processing based on experience replay technology is used for great data efficiency and convergence stability. Simulation experiments are presented to illustrate convergence stability, optimality and algorithmic performance of FR algorithm by comparison.

KW - Data episodes

KW - experience replay

KW - neural networks

KW - reinforcement learning (RL)

KW - uncertain systems

UR - https://www.scopus.com/pages/publications/85158843671

U2 - 10.1109/JAS.2023.123186

DO - 10.1109/JAS.2023.123186

M3 - 文章

AN - SCOPUS:85158843671

SN - 2329-9266

VL - 10

SP - 1288

EP - 1303

JO - IEEE/CAA Journal of Automatica Sinica

JF - IEEE/CAA Journal of Automatica Sinica

IS - 5

ER -

A Data-Based Feedback Relearning Algorithm for Uncertain Nonlinear Systems

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this