S2-AMNet: A lightweight Spatial–Spectral Adaptive Modulation Network for surface defect detection

Surface defect detection (SDD) has played an increasing role in quality inspection of industrial production and preventive maintenance of infrastructure. The existing deep learning methods mainly focus on designing special network architectures to execute specific defect detection tasks, and are also accompanied by high computational overhead and difficult practical deployment. This paper proposes a novel lightweight and efficient universal backbone network, namely Spatial–Spectral Adaptive Modulation Network (S2-AMNet), for multi-task SDD. Firstly, the spatial-adaptive modulation module (SAMM) with the adaptive dynamic salient central difference convolution (ADSCDC) and deformable axial convolution (DAC) is constructed for efficiently extracting multi-morphological features. Secondly, the adaptive frequency mixing module (AFMM) based on Fast Fourier Transform (FFT) is proposed to achieve global information mixing. Furthermore, the multi-path inverted bottleneck convolution (MPConv) is proposed to extract multi-scale features, and the Haar wavelet transform is introduced into downsampling layer to preserve the fine details and small-scale features. Finally, a special Multi-path Fusion Lite Segmentation (MF-LiteSeg) decoder is constructed for segmentation task. Extensive quantitative and qualitative comparisons, and ablation experiments have been performed on the custom and public datasets. Specifically, S2-AMNet achieves 100% Top-1 accuracy on 10-types steel surface defect (SSD-10) dataset, 81.5% mean average precision (mAP) at an intersection over union (IoU) threshold of 0.5 (mAP@0.5) on road damages detection of China (RDD-CHN) dataset and 86.41% mean IoU (mIoU) on road DeepCrack (RDC) dataset. The evaluation results demonstrate the favorable effect of various components in S2-AMNet and highlight the superior performance, robust adaptability, and promising the real-time recognition capability. The code and datasets are available at https://github.com/Baojy01/SSAM.