TY - GEN
T1 - Bugs in Modern LLM Agent Frameworks
T2 - ACM International Conference on the Foundations of Software Engineering, FSE 2026
AU - Zhu, Xinxue
AU - Wu, Jiacong
AU - Zhang, Xiaoyu
AU - Li, Tianlin
AU - Mu, Yanzhou
AU - Zhai, Juan
AU - Shen, Chao
AU - Fang, Chunrong
AU - Liu, Yang
N1 - Publisher Copyright:
© 2026 Copyright held by the owner/author(s).
PY - 2026/7/17
Y1 - 2026/7/17
N2 - LLM agents have been widely adopted in real-world applications, relying on agent frameworks for lifecycle execution and multiagent coordination. As these systems scale, understanding bugs in the underlying agent frameworks becomes critical. However, existing work mainly focuses on agent-level failures, overlooking framework-level bugs. To address this gap, we conduct an empirical study of 998 issue reports from CrewAI and LangGraph, and construct a taxonomy of 15 root causes and seven observable symptoms across five agent lifecycle stages: 'Agent Initialization', 'Perception', 'Self-Action', 'Mutual Interaction' and 'Evolution'. Our findings show that agent framework bugs mainly arise from 'API Misuse', 'API Incompatibility', and 'Documentation Desync', largely concentrated in the 'Self-Action' stage. Symptoms typically appear as 'Functional Error', 'Crash', and 'Build Failure', reflecting disruptions to task progression and control flow.
AB - LLM agents have been widely adopted in real-world applications, relying on agent frameworks for lifecycle execution and multiagent coordination. As these systems scale, understanding bugs in the underlying agent frameworks becomes critical. However, existing work mainly focuses on agent-level failures, overlooking framework-level bugs. To address this gap, we conduct an empirical study of 998 issue reports from CrewAI and LangGraph, and construct a taxonomy of 15 root causes and seven observable symptoms across five agent lifecycle stages: 'Agent Initialization', 'Perception', 'Self-Action', 'Mutual Interaction' and 'Evolution'. Our findings show that agent framework bugs mainly arise from 'API Misuse', 'API Incompatibility', and 'Documentation Desync', largely concentrated in the 'Self-Action' stage. Symptoms typically appear as 'Functional Error', 'Crash', and 'Build Failure', reflecting disruptions to task progression and control flow.
KW - bug taxonomy
KW - empirical study
KW - LLM agent framework
UR - https://www.scopus.com/pages/publications/105045832152
U2 - 10.1145/3803437.3805536
DO - 10.1145/3803437.3805536
M3 - 会议稿件
AN - SCOPUS:105045832152
T3 - FSE Companion 2026 - Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering
SP - 1588
EP - 1592
BT - FSE Companion 2026 - Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering
A2 - Tan, Shin Hwei
A2 - Khomh, Foutse
PB - Association for Computing Machinery, Inc
Y2 - 5 July 2026 through 9 July 2026
ER -