跳到主要导航 跳到搜索 跳到主要内容

Offline Multi-Agent Preference-Based Reinforcement Learning with Agent-aware Direct Preference Optimization

  • Qian Kou
  • , Mingyang Li
  • , Zeyang Liu
  • , Long Qian
  • , Zhuoran Chen
  • , Lipeng Wan
  • , Xingyu Chen
  • , Xuguang Lan
  • Xi'an Jiaotong University

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

1 引用 (Scopus)

摘要

Multi-agent Preference-Based Reinforcement Learning (MAPbRL) is promising in offline policy learning by leveraging human preferences to replace complex manual reward designing. Current MAPbRL methods use complicated structures to realize better reward modeling with off-the-shelf MARL algorithms and obtain the joint policy based on it. However, it faces a severe preference-behavior mismatch problem stemming from the instability of RL training and global-local preference inconsistency datasets in offline MARL, resulting in potential suboptimal policy convergence. To address this problem, we propose Agent-aware Multi-Agent Direct Preference Optimization (AMADPO) by utilizing a multi-agent preference predictor to guide agent-aware direct optimization from imbalanced preference labels, which can learn coordination policy from both positive and negative segments. Experimental results in SMAC environment show substantial improvements in global-local preference inconsistency datasets, demonstrating the effectiveness of AMADPO in solving the preference-behavior mismatch problem.

源语言英语
主期刊名Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2025
编辑Yevgeniy Vorobeychik, Sanmay Das, Ann Nowe
出版商International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
页1181-1190
页数10
ISBN(电子版)9798400714269
出版状态已出版 - 2025
活动24th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2025 - Detroit, 美国
期限: 19 5月 2025 → 23 5月 2025

丛书

姓名Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
ISSN(印刷版)1548-8403
ISSN(电子版)1558-2914

会议

会议24th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2025
国家/地区美国
市Detroit
时期19/05/25 → 23/05/25

学术指纹

探究 'Offline Multi-Agent Preference-Based Reinforcement Learning with Agent-aware Direct Preference Optimization' 的科研主题。它们共同构成独一无二的学术指纹。

引用此