Semi-supervised image rain removal using mutual consistency of rain kernel dictionaries

Single image rain removal is an important and challenging low-level visual task that benefits subsequent applications. The mainstream deep neural network (DNN)-based methods view this problem as an end-to-end supervised learning task using synthesized training datasets, which may suffer from generalization issues in real-world rainy scenarios. In this work, we propose a novel semi-supervised image deraining method by exploiting the intrinsic structure relationship between synthesized and real-world rain streaks. Specifically, we observe that synthesized and real-world rain may have different structures, but their underlying rain kernels can be represented by dictionaries with similar distributions. To this end, we minimize the discrepancy between synthesized and real-world rain kernel dictionaries (RKDs) with a novel consistency loss for semi-supervised training of DNN backbone. The mutual consistency between synthesized and real-world RKDs could bring more accurate characterization of the structure relationships between labeled and unlabeled data, and hence is beneficial to learn an effective and generalizable deraining network in a semi-supervised manner. To efficiently optimize the RKDs consistency-induced semi-supervised loss, we develop an alternating minimization-based algorithm. Extensive experiments on benchmark datasets such as R100L, Rain1400, Rain12, and real-world datasets demonstrate the effectiveness and generalization abilities of our method as compared with supervised and semi-supervised state-of-the-art image deraining methods.