PADiff: Reconstruction From Patch to Pixel With Normality-Guided Diffusion Model for Unsupervised Anomaly Localization

Anomaly localization (AL) is an indispensable and challenging task in manufacturing. Recently, diffusion models have been widely used to localize anomalies through discrepancies between original and reconstructed representations, which is based on the hypothesis that diffusion models regard anomalies as noise and reconstruct them to normal representations. However, anomalies usually deviate from prior standard Gaussian distribution and diffusion models cannot reconstruct anomaly parts as normal patterns well due to powerful generalization. These issues hinder the application of diffusion models in AL and lead to suboptimal performance. As a remedy, we present a novel framework for AL based on the diffusion model, dubbed PADiff. To enable the diffusion model to reconstruct abnormal regions to normal regions in an anomaly image, we propose to guide the diffusion model in the reconstruction process using its normal counterpart. High-quality guided normal counterpart plays a key role in our method. Therefore, we propose a patch-substitution strategy to obtain a high-quality-guided normal counterpart. Specifically, we first construct a normal patch memory bank using normal training samples. With a normal memory bank, we find potential anomaly patches in testing images and substitute them with most similar normal patches in the memory bank. After substitution, pseudo-normal images are generated to guide the diffusion model. To make our method more data-efficient, we divide an image into patches and propose patch-wise training and reconstruction. As one of our innovations, we propose to encode each patch into positional embedding and add it on time embedding, which introduces patch-level representation and position information in the diffusion model. Extensive experiments are conducted on three commonly used anomaly detection datasets (MVTec-AD, VisA, and BTAD) to showcase the state-of-the-art (SOTA) performance of the proposed PADiff. The source code is publicly available at https://github.com/Jay-zzcoder/padiff