An Energy-Efficient Accelerator for Rain Removal Based on Convolutional Neural Network

Rain removal is an important operation in intelligent visual surveillance systems. Many rain removal algorithms based on convolution neural networks (CNNs) are rarely deployed on resource constrained devices. One of the limiting factors is that memory access leads to high energy consumption. To reduce memory access during computation, previous works usually use a fixed computation pattern for different layers in CNN. For different and massive input and output feature maps, fixed computation pattern would lower the power efficiency. Thus, we propose a reconfigurable architecture to support different convolution mapping method. We use hybrid data reuse pattern to reduce energy consumption by 2.4-5.9 times over fixed computation pattern. The hardware is synthesized in the area of 9.83 mm2 at TSMC 65 nm technology and can restore a 1280×720 real world image in 0.57 s which achieves 56.1× and 2.1× speed-up comparing to CPU and GPU implementations. The simulation results show that the power efficiency is 287.7 GOPS/W running rain removal network.