Adaptive control of a marine vessel based on reinforcement learning

Zhao Yin; Wei He; Changyin Sun; Guang Li; Chenguang Yang

doi:10.23919/ChiCC.2018.8482656

Adaptive control of a marine vessel based on reinforcement learning

Zhao Yin
, Wei He
, Changyin Sun
, Guang Li
, Chenguang Yang

University of Science and Technology Beijing
Southeast University, Nanjing
Queen Mary University of London
South China University of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

In this paper, our main goal is to solve optimal control problem by using reinforcement learning (RL) algorithm for marine surface vessel system with known dynamic. And this algorithm is an optimal control algorithm based on policy iteration (PI), and it can obtain the suitable approximations of cost function and the optimized control policy. There are two neural networks (NNs), where critic NN aims to estimate the cost-to-go and actor NN is utilized to design suitable input controller and minimize the tracking error. A novel tuning method is given for critic NN and actor NN. The stability and convergence are proven by Lyapunov's direct method. Finally, the numerical simulations are conducted to demonstrate the feasibility and superiority of presented algorithm.

Original language	English
Title of host publication	Proceedings of the 37th Chinese Control Conference, CCC 2018
Editors	Xin Chen, Qianchuan Zhao
Publisher	IEEE Computer Society
Pages	2735-2740
Number of pages	6
ISBN (Electronic)	9789881563941
DOIs	https://doi.org/10.23919/ChiCC.2018.8482656
State	Published - 5 Oct 2018
Externally published	Yes
Event	37th Chinese Control Conference, CCC 2018 - Wuhan, China Duration: 25 Jul 2018 → 27 Jul 2018

Publication series

Name	Chinese Control Conference, CCC
Volume	2018-July
ISSN (Print)	1934-1768
ISSN (Electronic)	2161-2927

Conference

Conference	37th Chinese Control Conference, CCC 2018
Country/Territory	China
City	Wuhan
Period	25/07/18 → 27/07/18

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 14 Life Below Water

Keywords

Actor Neural Networks
Critic Neural Networks
Lyapunov Method
Marine Vessel
Reinforcement Learning

Access to Document

10.23919/ChiCC.2018.8482656

Cite this

@inproceedings{60caf97cd6bc4c3b993874ff044785b5,

title = "Adaptive control of a marine vessel based on reinforcement learning",

abstract = "In this paper, our main goal is to solve optimal control problem by using reinforcement learning (RL) algorithm for marine surface vessel system with known dynamic. And this algorithm is an optimal control algorithm based on policy iteration (PI), and it can obtain the suitable approximations of cost function and the optimized control policy. There are two neural networks (NNs), where critic NN aims to estimate the cost-to-go and actor NN is utilized to design suitable input controller and minimize the tracking error. A novel tuning method is given for critic NN and actor NN. The stability and convergence are proven by Lyapunov's direct method. Finally, the numerical simulations are conducted to demonstrate the feasibility and superiority of presented algorithm.",

keywords = "Actor Neural Networks, Critic Neural Networks, Lyapunov Method, Marine Vessel, Reinforcement Learning",

author = "Zhao Yin and Wei He and Changyin Sun and Guang Li and Chenguang Yang",

note = "Publisher Copyright: {\textcopyright} 2018 Technical Committee on Control Theory, Chinese Association of Automation.; 37th Chinese Control Conference, CCC 2018 ; Conference date: 25-07-2018 Through 27-07-2018",

year = "2018",

month = oct,

day = "5",

doi = "10.23919/ChiCC.2018.8482656",

language = "英语",

series = "Chinese Control Conference, CCC",

publisher = "IEEE Computer Society",

pages = "2735--2740",

editor = "Xin Chen and Qianchuan Zhao",

booktitle = "Proceedings of the 37th Chinese Control Conference, CCC 2018",

}

Yin, Z, He, W, Sun, C, Li, G & Yang, C 2018, Adaptive control of a marine vessel based on reinforcement learning. in X Chen & Q Zhao (eds), Proceedings of the 37th Chinese Control Conference, CCC 2018., 8482656, Chinese Control Conference, CCC, vol. 2018-July, IEEE Computer Society, pp. 2735-2740, 37th Chinese Control Conference, CCC 2018, Wuhan, China, 25/07/18. https://doi.org/10.23919/ChiCC.2018.8482656

Adaptive control of a marine vessel based on reinforcement learning. / Yin, Zhao; He, Wei; Sun, Changyin et al.
Proceedings of the 37th Chinese Control Conference, CCC 2018. ed. / Xin Chen; Qianchuan Zhao. IEEE Computer Society, 2018. p. 2735-2740 8482656 (Chinese Control Conference, CCC; Vol. 2018-July).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Adaptive control of a marine vessel based on reinforcement learning

AU - Yin, Zhao

AU - He, Wei

AU - Sun, Changyin

AU - Li, Guang

AU - Yang, Chenguang

PY - 2018/10/5

Y1 - 2018/10/5

N2 - In this paper, our main goal is to solve optimal control problem by using reinforcement learning (RL) algorithm for marine surface vessel system with known dynamic. And this algorithm is an optimal control algorithm based on policy iteration (PI), and it can obtain the suitable approximations of cost function and the optimized control policy. There are two neural networks (NNs), where critic NN aims to estimate the cost-to-go and actor NN is utilized to design suitable input controller and minimize the tracking error. A novel tuning method is given for critic NN and actor NN. The stability and convergence are proven by Lyapunov's direct method. Finally, the numerical simulations are conducted to demonstrate the feasibility and superiority of presented algorithm.

AB - In this paper, our main goal is to solve optimal control problem by using reinforcement learning (RL) algorithm for marine surface vessel system with known dynamic. And this algorithm is an optimal control algorithm based on policy iteration (PI), and it can obtain the suitable approximations of cost function and the optimized control policy. There are two neural networks (NNs), where critic NN aims to estimate the cost-to-go and actor NN is utilized to design suitable input controller and minimize the tracking error. A novel tuning method is given for critic NN and actor NN. The stability and convergence are proven by Lyapunov's direct method. Finally, the numerical simulations are conducted to demonstrate the feasibility and superiority of presented algorithm.

KW - Actor Neural Networks

KW - Critic Neural Networks

KW - Lyapunov Method

KW - Marine Vessel

KW - Reinforcement Learning

UR - https://www.scopus.com/pages/publications/85056118512

U2 - 10.23919/ChiCC.2018.8482656

DO - 10.23919/ChiCC.2018.8482656

M3 - 会议稿件

AN - SCOPUS:85056118512

T3 - Chinese Control Conference, CCC

SP - 2735

EP - 2740

BT - Proceedings of the 37th Chinese Control Conference, CCC 2018

A2 - Chen, Xin

A2 - Zhao, Qianchuan

PB - IEEE Computer Society

T2 - 37th Chinese Control Conference, CCC 2018

Y2 - 25 July 2018 through 27 July 2018

ER -

Adaptive control of a marine vessel based on reinforcement learning

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this