Design an Efficient DNN Inference Framework with PS-PL Synergies in FPGA for Edge Computing

in recent years, Deep Neural Network (DNN) based methods have achieved great success for changing inflexible machine to intelligent and live system. Internet of Things (IoT) applications equipped with DNN in domains such as self-driving, speech processing and video surveillance particular challenges. Since DNNs are memory-/computing- intensive applications, FPGAs are customized to accelerate DNN on edge system due to their low latency and high energy efficiency. However, embedded CPU is usually essential for IoT system and single accelerator of FPGA is hard to gain maximized performance of the whole IoT system. Therefore, we design an efficient DNN inference framework with programming system (PS, e.g., ARM CPU) and programming logic (PL, e.g. FPGA) synergies for intelligent IoT system. Compared with the common IoT system only separately leverages FPGA for DNN accelerating and ARM for other tasks, our framework can make ARM and FPGA jointly accelerate DNN inference from the beginning. Our PS-PL co-design approach can take full advantage of the merits of ARM (control operations) and FPGA (parallel computing operations) for various DNN model accelerating. We implemented our co-design framework on Zynq SoCs. The experimental results show that our framework can achieve maximal 205.9 GOPs/s DNN inference accelerating under 6.2W while it would not affect other CPU tasks running on IoT system.