跳到主要导航 跳到搜索 跳到主要内容

IsoSched: Preemptive Tile Cascaded Scheduling of Multi-DNN via Subgraph Isomorphism

  • Boran Zhao
  • , Zihang Yuan
  • , Hetian Liu
  • , Yanbin Hu
  • , Haiming Zhai
  • , Haoruo Zhang
  • , Wenzhe Zhao
  • , Tian Xia
  • , Pengju Ren
  • School of Software Engineering
  • Xi'an Jiaotong University
  • Nanyang Technological University

科研成果: 期刊稿件 › 文章 › 同行评审

摘要

Deploying deep neural network (DNN) accelerators with Layer Temporal Scheduling (LTS) often incurs significant overheads (e.g., energy and latency), as intermediate activations must be cached in DRAM. To alleviate this issue, Tile Spatial Scheduling (TSS) has been proposed, which reduces such costs by fragmenting inter-layer data into smaller tiles communicated via on-chip link. However, many emerging applications require concurrently executing multiple DNNs with complex topologies, where certain critical tasks must preempt other tasks to meet stringent latency requirements (e.g., in autonomous driving, obstacle detection must be completed on the order of tens of milliseconds). Unfortunately, existing TSS works lack support for preemptive execution, while prior preemption schemes are based on LTS and therefore inherit its costly overheads. This observation highlights the necessity of preemptive and efficient TSS-based frameworks. Nevertheless, designing such a system presents significant challenges, particularly due to the requirement of preemption in graphs with complex topologies (e.g., in modern large language models, the number of graph edges can scale to tens of thousands). To tackle this challenge, we introduce IsoSched, the first framework that enables preemptive multi-DNN scheduling on TSS architecture. IsoSched firstly formulates preemptive scheduling of graphs with complex topologies as an integer-linear program (ILP) and a subgraph isomorphism problem; secondly, we apply Layer Concatenate and Split (LCS) for load balancing of tile pipeline; thirdly, to enable efficient subgraph matching, we integrate an Ullmann-based algorithm enhanced with Monte Carlo Tree Search (MCTS) to accelerate the search process, while leveraging compact matrix encoding (i.e., Compressed Sparse Row, CSR) to reduce memory overhead. IsoSched outperforms existing LTS-PRM approaches (i.e., PREMA, Planaria, CD-MSA, MoCA) in Latency-Bound Throughput (LBT), speedup, and energy efficiency, and achieves higher critical task satisfaction rates than TSS-NPRM (i.e., HASP) across varying task complexities.

联合国可持续发展目标

此成果有助于实现下列可持续发展目标:

  1. 可持续发展目标 7 - 经济适用的清洁能源
    可持续发展目标 7 经济适用的清洁能源

学术指纹

探究 'IsoSched: Preemptive Tile Cascaded Scheduling of Multi-DNN via Subgraph Isomorphism' 的科研主题。它们共同构成独一无二的学术指纹。

引用此