Deep Learning on Mobile and Embedded Devices: State-of-the-art, Challenges, and Future Directions

Recent years have witnessed an exponential increase in the use of mobile and embedded devices. With the great success of deep learning in many fields, there is an emerging trend to deploy deep learning on mobile and embedded devices to better meet the requirement of real-time applications and user privacy protection. However, the limited resources of mobile and embedded devices make it challenging to fulfill the intensive computation and storage demand of deep learning models. In this survey, we conduct a comprehensive review on the related issues for deep learning on mobile and embedded devices. We start with a brief introduction of deep learning and discuss major challenges of implementing deep learning models on mobile and embedded devices. We then conduct an in-depth survey on important compression and acceleration techniques that help adapt deep learning models to mobile and embedded devices, which we specifically classify as pruning, quantization, model distillation, network design strategies, and low-rank factorization. We elaborate on the hardware-based solutions, including mobile GPU, FPGA, and ASIC, and describe software frameworks for mobile deep learning models, especially the development of frameworks based on OpenCL and RenderScript. After that, we present the application of mobile deep learning in a variety of areas, such as navigation, health, speech recognition, and information security. Finally, we discuss some future directions for deep learning on mobile and embedded devices to inspire further research in this area.