Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report

Andrey Ignatov; Radu Timofte; Cheng Ming Chiang; Hsien Kai Kuo; Yu Syuan Xu; Man Yu Lee; Allen Lu; Chia Ming Cheng; Chih Cheng Chen; Jia Ying Yong; Hong Han Shuai; Wen Huang Cheng; Zhuang Jia; Tianyu Xu; Yijian Zhang; Long Bao; Heng Sun; Diankai Zhang; Si Gao; Shaoli Liu; Biao Wu; Xiaofeng Zhang; Chengjian Zheng; Kaidi Lu; Ning Wang; Xiao Sun; Hao Dong Wu; Xuncheng Liu; Weizhan Zhang; Caixia Yan; Haipeng Du; Qinghua Zheng; Qi Wang; Wangdu Chen; Ran Duan; Mengdi Sun; Dan Zhu; Guannan Chen; Hojin Cho; Steve Kim; Shijie Yue; Chenghua Li; Zhengyang Zhuge; Wei Chen; Wenxu Wang; Yufeng Zhou; Xiaochen Cai; Hengxing Cai; Kele Xu; Li Liu; Zehua Cheng; Wenyi Lian; Wenjing Lian

doi:10.1007/978-3-031-25066-8_6

Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report

Andrey Ignatov
, Radu Timofte
, Cheng Ming Chiang
, Hsien Kai Kuo
, Yu Syuan Xu
, Man Yu Lee
, Allen Lu
, Chia Ming Cheng
, Chih Cheng Chen
, Jia Ying Yong
, Hong Han Shuai
, Wen Huang Cheng
, Zhuang Jia
, Tianyu Xu
, Yijian Zhang
, Long Bao
, Heng Sun
, Diankai Zhang
, Si Gao
, Shaoli Liu

Biao Wu, Xiaofeng Zhang, Chengjian Zheng, Kaidi Lu, Ning Wang, Xiao Sun, Hao Dong Wu, Xuncheng Liu, Weizhan Zhang, Caixia Yan, Haipeng Du, Qinghua Zheng, Qi Wang, Wangdu Chen, Ran Duan, Mengdi Sun, Dan Zhu, Guannan Chen, Hojin Cho, Steve Kim, Shijie Yue, Chenghua Li, Zhengyang Zhuge, Wei Chen, Wenxu Wang, Yufeng Zhou, Xiaochen Cai, Hengxing Cai, Kele Xu, Li Liu, Zehua Cheng, Wenyi Lian, Wenjing Lian

电子与信息学部

Swiss Federal Institute of Technology Zurich
AI Witchlabs
University of Würzburg
MediaTek
National Yang Ming Chiao Tung University
Xiaomi
ZTE Corporation
Xi'an Jiaotong University
Ltd.
Boe Technology Group
GenGenAI
North China University of Technology
CAS - Institute of Automation
CAS - Institute of Computing Technology
4Paradigm Inc.
National University of Defense Technology
University of Oxford
Uppsala University
Northeastern University China

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

6 引用（Scopus）

摘要

Video super-resolution is one of the most popular tasks on mobile devices, being widely used for an automatic improvement of low-bitrate and low-resolution video streams. While numerous solutions have been proposed for this problem, they are usually quite computationally demanding, demonstrating low FPS rates and power efficiency on mobile devices. In this Mobile AI challenge, we address this problem and propose the participants to design an end-to-end real-time video super-resolution solution for mobile NPUs optimized for low energy consumption. The participants were provided with the REDS training dataset containing video sequences for a 4X video upscaling task. The runtime and power efficiency of all models was evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit capable of accelerating floating-point and quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt/30 FPS] power consumption. A detailed description of all models developed in the challenge is provided in this paper.

源语言	英语
主期刊名	Computer Vision – ECCV 2022 Workshops, Proceedings
编辑	Leonid Karlinsky, Tomer Michaeli, Ko Nishino
出版商	Springer Science and Business Media Deutschland GmbH
页	130-152
页数	23
ISBN（印刷版）	9783031250651
DOI	https://doi.org/10.1007/978-3-031-25066-8_6
出版状态	已出版 - 2023
活动	Workshops held at the 17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, 以色列期限: 23 10月 2022 → 27 10月 2022

出版系列

姓名	Lecture Notes in Computer Science
卷	13803 LNCS
ISSN（印刷版）	0302-9743
ISSN（电子版）	1611-3349

会议

会议	Workshops held at the 17th European Conference on Computer Vision, ECCV 2022
国家/地区	以色列
市	Tel Aviv
时期	23/10/22 → 27/10/22

联合国可持续发展目标

此成果有助于实现下列可持续发展目标：

可持续发展目标 7 经济适用的清洁能源

访问文件

10.1007/978-3-031-25066-8_6

其它文件与链接

链接到 Scopus 的出版物

引用此

Ignatov, A., Timofte, R., Chiang, C. M., Kuo, H. K., Xu, Y. S., Lee, M. Y., Lu, A., Cheng, C. M., Chen, C. C., Yong, J. Y., Shuai, H. H., Cheng, W. H., Jia, Z., Xu, T., Zhang, Y., Bao, L., Sun, H., Zhang, D., Gao, S., ... Lian, W. (2023). Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report. 在 L. Karlinsky, T. Michaeli, & K. Nishino (编辑), Computer Vision – ECCV 2022 Workshops, Proceedings (页码 130-152). (Lecture Notes in Computer Science; 卷 13803 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-25066-8_6

Ignatov, Andrey ; Timofte, Radu ; Chiang, Cheng Ming 等. / Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 Challenge : Report. Computer Vision – ECCV 2022 Workshops, Proceedings. 编辑 / Leonid Karlinsky ; Tomer Michaeli ; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. 页码 130-152 (Lecture Notes in Computer Science).

@inproceedings{f61f1158e6c844759ae516836c404c58,

title = "Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI \& AIM 2022 Challenge: Report",

abstract = "Video super-resolution is one of the most popular tasks on mobile devices, being widely used for an automatic improvement of low-bitrate and low-resolution video streams. While numerous solutions have been proposed for this problem, they are usually quite computationally demanding, demonstrating low FPS rates and power efficiency on mobile devices. In this Mobile AI challenge, we address this problem and propose the participants to design an end-to-end real-time video super-resolution solution for mobile NPUs optimized for low energy consumption. The participants were provided with the REDS training dataset containing video sequences for a 4X video upscaling task. The runtime and power efficiency of all models was evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit capable of accelerating floating-point and quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt/30 FPS] power consumption. A detailed description of all models developed in the challenge is provided in this paper.",

keywords = "AI Benchmark, Deep learning, MediaTek, Mobile AI, Mobile AI challenge, Mobile NPUs, Video super-resolution",

author = "Andrey Ignatov and Radu Timofte and Chiang, \{Cheng Ming\} and Kuo, \{Hsien Kai\} and Xu, \{Yu Syuan\} and Lee, \{Man Yu\} and Allen Lu and Cheng, \{Chia Ming\} and Chen, \{Chih Cheng\} and Yong, \{Jia Ying\} and Shuai, \{Hong Han\} and Cheng, \{Wen Huang\} and Zhuang Jia and Tianyu Xu and Yijian Zhang and Long Bao and Heng Sun and Diankai Zhang and Si Gao and Shaoli Liu and Biao Wu and Xiaofeng Zhang and Chengjian Zheng and Kaidi Lu and Ning Wang and Xiao Sun and Wu, \{Hao Dong\} and Xuncheng Liu and Weizhan Zhang and Caixia Yan and Haipeng Du and Qinghua Zheng and Qi Wang and Wangdu Chen and Ran Duan and Mengdi Sun and Dan Zhu and Guannan Chen and Hojin Cho and Steve Kim and Shijie Yue and Chenghua Li and Zhengyang Zhuge and Wei Chen and Wenxu Wang and Yufeng Zhou and Xiaochen Cai and Hengxing Cai and Kele Xu and Li Liu and Zehua Cheng and Wenyi Lian and Wenjing Lian",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; Workshops held at the 17th European Conference on Computer Vision, ECCV 2022 ; Conference date: 23-10-2022 Through 27-10-2022",

year = "2023",

doi = "10.1007/978-3-031-25066-8\_6",

language = "英语",

isbn = "9783031250651",

series = "Lecture Notes in Computer Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "130--152",

editor = "Leonid Karlinsky and Tomer Michaeli and Ko Nishino",

booktitle = "Computer Vision – ECCV 2022 Workshops, Proceedings",

}

Ignatov, A, Timofte, R, Chiang, CM, Kuo, HK, Xu, YS, Lee, MY, Lu, A, Cheng, CM, Chen, CC, Yong, JY, Shuai, HH, Cheng, WH, Jia, Z, Xu, T, Zhang, Y, Bao, L, Sun, H, Zhang, D, Gao, S, Liu, S, Wu, B, Zhang, X, Zheng, C, Lu, K, Wang, N, Sun, X, Wu, HD, Liu, X, Zhang, W, Yan, C, Du, H, Zheng, Q, Wang, Q, Chen, W, Duan, R, Sun, M, Zhu, D, Chen, G, Cho, H, Kim, S, Yue, S, Li, C, Zhuge, Z, Chen, W, Wang, W, Zhou, Y, Cai, X, Cai, H, Xu, K, Liu, L, Cheng, Z, Lian, W & Lian, W 2023, Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report. 在 L Karlinsky, T Michaeli & K Nishino (编辑), Computer Vision – ECCV 2022 Workshops, Proceedings. Lecture Notes in Computer Science, 卷 13803 LNCS, Springer Science and Business Media Deutschland GmbH, 页码 130-152, Workshops held at the 17th European Conference on Computer Vision, ECCV 2022, Tel Aviv, 以色列, 23/10/22. https://doi.org/10.1007/978-3-031-25066-8_6

Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report. / Ignatov, Andrey; Timofte, Radu; Chiang, Cheng Ming 等.
Computer Vision – ECCV 2022 Workshops, Proceedings. 编辑 / Leonid Karlinsky; Tomer Michaeli; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. 页码 130-152 (Lecture Notes in Computer Science; 卷 13803 LNCS).

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

TY - GEN

T1 - Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 Challenge

T2 - Workshops held at the 17th European Conference on Computer Vision, ECCV 2022

AU - Ignatov, Andrey

AU - Timofte, Radu

AU - Chiang, Cheng Ming

AU - Kuo, Hsien Kai

AU - Xu, Yu Syuan

AU - Lee, Man Yu

AU - Lu, Allen

AU - Cheng, Chia Ming

AU - Chen, Chih Cheng

AU - Yong, Jia Ying

AU - Shuai, Hong Han

AU - Cheng, Wen Huang

AU - Jia, Zhuang

AU - Xu, Tianyu

AU - Zhang, Yijian

AU - Bao, Long

AU - Sun, Heng

AU - Zhang, Diankai

AU - Gao, Si

AU - Liu, Shaoli

AU - Wu, Biao

AU - Zhang, Xiaofeng

AU - Zheng, Chengjian

AU - Lu, Kaidi

AU - Wang, Ning

AU - Sun, Xiao

AU - Wu, Hao Dong

AU - Liu, Xuncheng

AU - Zhang, Weizhan

AU - Yan, Caixia

AU - Du, Haipeng

AU - Zheng, Qinghua

AU - Wang, Qi

AU - Chen, Wangdu

AU - Duan, Ran

AU - Sun, Mengdi

AU - Zhu, Dan

AU - Chen, Guannan

AU - Cho, Hojin

AU - Kim, Steve

AU - Yue, Shijie

AU - Li, Chenghua

AU - Zhuge, Zhengyang

AU - Chen, Wei

AU - Wang, Wenxu

AU - Zhou, Yufeng

AU - Cai, Xiaochen

AU - Cai, Hengxing

AU - Xu, Kele

AU - Liu, Li

AU - Cheng, Zehua

AU - Lian, Wenyi

AU - Lian, Wenjing

PY - 2023

Y1 - 2023

N2 - Video super-resolution is one of the most popular tasks on mobile devices, being widely used for an automatic improvement of low-bitrate and low-resolution video streams. While numerous solutions have been proposed for this problem, they are usually quite computationally demanding, demonstrating low FPS rates and power efficiency on mobile devices. In this Mobile AI challenge, we address this problem and propose the participants to design an end-to-end real-time video super-resolution solution for mobile NPUs optimized for low energy consumption. The participants were provided with the REDS training dataset containing video sequences for a 4X video upscaling task. The runtime and power efficiency of all models was evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit capable of accelerating floating-point and quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt/30 FPS] power consumption. A detailed description of all models developed in the challenge is provided in this paper.

AB - Video super-resolution is one of the most popular tasks on mobile devices, being widely used for an automatic improvement of low-bitrate and low-resolution video streams. While numerous solutions have been proposed for this problem, they are usually quite computationally demanding, demonstrating low FPS rates and power efficiency on mobile devices. In this Mobile AI challenge, we address this problem and propose the participants to design an end-to-end real-time video super-resolution solution for mobile NPUs optimized for low energy consumption. The participants were provided with the REDS training dataset containing video sequences for a 4X video upscaling task. The runtime and power efficiency of all models was evaluated on the powerful MediaTek Dimensity 9000 platform with a dedicated AI processing unit capable of accelerating floating-point and quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 500 FPS rate and 0.2 [Watt/30 FPS] power consumption. A detailed description of all models developed in the challenge is provided in this paper.

KW - AI Benchmark

KW - Deep learning

KW - MediaTek

KW - Mobile AI

KW - Mobile AI challenge

KW - Mobile NPUs

KW - Video super-resolution

UR - https://www.scopus.com/pages/publications/85151167744

U2 - 10.1007/978-3-031-25066-8_6

DO - 10.1007/978-3-031-25066-8_6

M3 - 会议稿件

AN - SCOPUS:85151167744

SN - 9783031250651

T3 - Lecture Notes in Computer Science

SP - 130

EP - 152

BT - Computer Vision – ECCV 2022 Workshops, Proceedings

A2 - Karlinsky, Leonid

A2 - Michaeli, Tomer

A2 - Nishino, Ko

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 23 October 2022 through 27 October 2022

ER -

Ignatov A, Timofte R, Chiang CM, Kuo HK, Xu YS, Lee MY 等. Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report. 在 Karlinsky L, Michaeli T, Nishino K, 编辑, Computer Vision – ECCV 2022 Workshops, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. 页码 130-152. (Lecture Notes in Computer Science). doi: 10.1007/978-3-031-25066-8_6

Power Efficient Video Super-Resolution on Mobile NPUs with Deep Learning, Mobile AI & AIM 2022 Challenge: Report

摘要

出版系列

会议

联合国可持续发展目标

访问文件

其它文件与链接

学术指纹

引用此