Semantic Distribution and Authenticity Discrepancy Alignment for AI-Generated Image Detection

Generative models have achieved remarkable success in producing vivid images. Compared with real images, generated ones still show different semantic structures that features with different semantic classes collapse as a single cluster. Pioneer works leverage the discrepancy of semantic structure in fixed high-level semantic feature space to identify forgery images. Nevertheless, such frozen pre-trained representation models are insensitive to subtle forgery traces. Meanwhile, vanilla fine-tuning methods can distort the pre-trained semantic knowledge and collapse to the real-fake binary distribution, losing generalization capability in newly emerged generative models. In this paper, we propose the semantic distribution and authenticity discrepancy alignment algorithm (STERM), which learns high-level semantic structures of real-world categories and low-level forgery traces for detecting AI-generated images from unseen generative models and frameworks. Specifically, we first capture semantic features of images by the frozen CLIP and further extract forgery features by a forgery encoder. Then, we propose semantic distribution alignment (SDA) to align the semantic structure of real-world categories by enforcing forgery feature distribution shifting towards the semantic feature space. Next, we introduce authenticity discrepancy alignment (ADA) to minimize the authenticity discrepancy between forgery and semantic features, constraining forgery features from collapsing into the source domain-biased distribution and learning the semantic structure of real-world categories. Extensive experiments on GAN-based and diffusion model-based datasets demonstrate the generalization capability of the proposed method.