Pretender: Universal Active Defense against Diffusion Finetuning Attacks

The proliferation of Diffusion Models (DMs) has marked a significant advancement in AI-generated image creation. However, this success has also spawned a new form of infringement threat termed the Diffusion Finetuning Attack (DFA), where malicious attackers can finetune pre-trained DMs using minimal resources to illicitly synthesize copyright-infringing images by ‘stealing’ information from personal photographic data or artwork, raising critical concerns about privacy and intellectual property rights. Recognizing the limitations of current defense strategies, which exhibit inadequate generalizability and suboptimal mechanism efficacy, we introduce an universal and effective active defense mechanism that applies subtle protective noise to images, guarding against information theft from DFAs. Our work innovatively conceptualizes active defense as a bi-level optimization problem, focusing on attackers’ common behaviors to enhance the generalization of defense. Guided by this optimization framework, we have developed a novel algorithm named Pretender, where we adversarially trained a surrogate model to facilitate the generation of more effective protective noise. In addition, a Simultaneous Gradient Back-Propagation (SGBP) technique is introduced to significantly enhance computational efficiency. Extensive experiments including real-world evaluations have demonstrated the effectiveness of Pretender. By applying minimal perturbations (p = 0.03), Pretender successfully disrupted the quality and semantics of images synthesized by diverse DFAs, achieving a comprehensive and prominent improvement in various automated evaluation metrics by 22.27% and in human assessment scores by 94.28%.