Hierarchical frequency adaptation for all-in-one image restoration

Effective degradation-aware adaptation is critical for all-in-one image restoration. However, prevailing methods typically model degradations solely in the spatial domain, often neglecting the more discriminative information inherent in the frequency spectrum. To address this limitation, we propose a Hierarchical Frequency-driven Network (HFre-Net) that first establishes a global frequency context to determine what to restore, and then refines how to restore it through layer-wise frequency expert specialization. At the global level, a Frequency-Prompted Modulation (FPM) module analyzes the input frequency spectrum to generate a unified prompt that provides top-down, degradation-aware guidance for the entire restoration pipeline. At the local level, the Mixture-of-Frequency-Experts (MoFE) is integrated into each decoder layer to determine how features should be restored. MoFE dynamically routes layer-wise features to degradation-specific frequency experts for adaptive computation tailored to the evolving degradation characteristics, while a shared expert models task-invariant representations. The synergy between global frequency prompting and layer-wise expert specialization enables precise and robust restoration across diverse degradations. Extensive experiments on multiple benchmarks demonstrate that our method shows superior performance, validating the effectiveness of our hierarchical frequency adaptation framework. The code will be released at https://github.com/wuyang2691/HFre-Net.