HQ-IRN: Quantizing High-Frequency Features for Image Rescaling

Saving and transmitting high-resolution (HR) images are often demanded in real life, especially in social media applications. The recently developed image rescaling techniques provide a storage and transmission economic way to deal with this problem, by jointly learning the downscaling and upscaling mappings for the images with the aid of the invertible neural network (INN). In the original pipeline, the high-frequency information is generally discarded in the downscaled low-resolution (LR) image, while is randomly sampled when doing upscaling, so that the storage and transmission cost can be minimal. However, the quality of the reconstructed HR image is limited due to the ignorance of the high-frequency information, and thus there are researchers trying to improve the upscaling performance by paying a bit more storage to partially save the high-frequency features. In this work, following this research line, we propose a new strategy to improve the image rescaling performance by more efficiently utilizing additional storage. Specifically, instead of saving the partial high-frequency features, we propose to quantize those features with a learned codebook and save the corresponding index matrix. Such a vector quantization strategy can recover as much as possible high-frequency features, and thus leads to a better image rescaling performance. Besides, the additional storage cost is the same or can be even less compared with existing methods. Experiments on a series of benchmark datasets demonstrate the effectiveness of the proposed method against current state-of-the-art ones.