2.1. CNN-Based CG Image Detection

In recent years, deep learning–based methods have gradually played a more important role in image forensics research than traditional methods based on hand-crafted features. A growing number of CNN-based methods have been developed to address CG image forensics. Previous CNN-based methods rarely deviate from a local-to-global manner [7–9, 12, 13, 16, 17]. CNNs are applied to obtain local decisions on a certain number of image patches that are cropped from a full-size image, and a global decision can be obtained via a majority vote scheme. Therefore, the performance improvement of these methods may come at the cost of high computational overhead.

More recently, CNN-based methods have been developed for CG image forensics based on an entire image or image patch, and the detection performances are improved by designing network frameworks to explore effective distinct features between CG and PG images. Several studies [14, 18] have developed dual-branch networks to learn RGB-based distinct features and high frequency–based distinct features of CG and PG images simultaneously. Bai et al. [10] designed a texture-aware network to perform texture enhancement on features and learn the relations of different feature channels. Furthermore, considering the existing datasets of CG image forensics being constructed in former times and limited in both quantity and diversity, they built a new complex and diverse benchmark for the CG forensics task, i.e., a large-scale CG images benchmark (LSCGB). Gangan et al. [19] explored discriminative features in different color spaces, RGB, LCH, and HSV by parallelly fusing three EfficientNet networks [20].

Yao et al. [21] proposed a CG image detection method by extracting both the shallow and the deep semantic features of the image which are beneficial to the task of CG image forensics. Ju et al. [22] proposed a global and local feature fusion (GLFF) framework to learn multiscale global features and refined local features for detecting AI-generated images.

Existing deep learning–based methods have demonstrated promising performance in CG image forensics, but their generalization capability and robustness remain limited. This is because designing effective network architectures for CG image detection often requires extensive domain knowledge and experience. In addition, these methods typically focus on specific discrepancies between CG and PG images. Moreover, these existing CNNs used for CG image detection are trained only by the supervision of CE loss, leading to insufficient learning of intrinsic forensic features. In this work, we proposed a supervised contrastive learning framework for CG image forensics to adaptively learn intrinsic forensic features and further achieve high generalization capability and robustness.

2.2. Contrastive Learning

Contrastive learning has come to researchers’ notice in recent years because of its philosophy of maximizing the mutual information between two transformation views of the same image [23]. The transformation views contribute to learning transformation invariant features that can resist perturbations [24]. The traditional contrastive learning works [25–28] primarily employed self-supervised pretraining by designing pretext tasks, and the pretrained models were able to achieve good performance when fine-tuned on downstream tasks. Promising transfer ability on downstream tasks fully demonstrates its efficiency in learning general representation.

Khosla et al. [29] first conducted research on supervised contrastive learning in tasks such as detection, classification, and segmentation, demonstrating that models trained end-to-end with supervised contrastive learning outperformed unsupervised pretrained models that were fine-tuned on specific downstream tasks. Consequently, researchers [30, 31] applied supervised contrastive learning in the deepfake forensics task. Compared to the models solely based on CE loss, the proposed InfoNCE loss has proven to be beneficial in learning the general discriminative features. In addition, recent advancements in image tampering detection [32–34] have showcased that contrastive learning facilitated learning intrinsic features, which emphasize the dissimilarity between authentic and manipulated regions. By leveraging the relationships of local features between tampered and authentic regions for discriminative representation learning, the performance of image tampering detection can be highly improved even in the presence of disturbance. The recent CG image detection work [35] has introduced self-supervised learning by designing pretext tasks. However, there was still a performance gap when compared with a state-of-the-art fully supervised method.

Previous contrastive learning–based works were mostly aimed at image tampering detection or deepfake detection, which may not be suitable for CG image forensics tasks. Based on this, we designed a FCL framework. First, we designed a FAM to extract spatial and frequency domain features that are more conducive to forensics. Subsequently, we applied supervision to the deep and shallow layers of the backbone network to comprehensively learn the inconsistencies between CG and PG images from both global and local perspectives.

3.1. FAM

In our FCL framework, data augmentation plays a vital role in capturing general forensic cues. Our specially designed FAM serves the purpose of eliminating task-irrelevant factors within training samples and uncovering more discriminative information from both spatial and frequency domains. As depicted in Figure 1, we obtained two different views of the original image via random cropping. These views are then subjected to spatial or frequency augmentation to generate the inputs for contrastive learning.

3.1.2. Frequency Augmentation

In previous studies [13, 14, 37, 38], a common insight is to prevent models from learning semantic information, while simultaneously promoting the learning of nonsemantic information, such as sensor noise patterns. Inspired by this observation, we develop a semantic-agnostic augmentation in the frequency domain, termed Phase Mix-up. Based on the assumption that Fourier phase information predominantly contains semantic information and is deemed less significant for forensic tasks [39], the proposed frequency augmentation method mixes up the phase information between CG-CG or PG-PG image pairs while keeping their spectral amplitudes unchanged, thereby effectively uncovering critical forensic cues in the frequency domain. Take a CG image as an example, the image patches are first transformed to the frequency domain by using fast Fourier transform (FFT). It can be noted as follows:

$$ ()

where $$ is an image patch randomly cropped from the current CG image. $$ is an image patch randomly selected image from another CG image in the current training batch. A_φ and P_φ represent the amplitude spectrum and phase spectrum in the frequency domain, respectively.

Then, the process of obtaining the mixed phase spectrum can be formulated as

$$ ()

where P_cg1 and P_cg2 are phase spectrum of $$ and $$, respectively. $$ is the mixed phase spectrum of $$. β is the mixing coefficient.

The frequency-augmented image of $$, denoted as V_fre(x_cg), can be obtained by inverse FFT (iFFT). It can be noted as

$$ ()

Figure 2 gives the pipeline of our proposed FAM. By applying spatial and frequency augmentation in different branches, the positive pair (V_spa(x), V_fre(x)) exhibits different high-level semantic content but still shares similar amplitude spectrum information. By pulling close the distance between the spatial and frequency views of the same image, the model can effectively learn more critical forensics information to boost its generalization performance.

3.2. Pipeline of FCL Framework

3.3. PCL Module

Since it is still challenging to replicate the complex textures found in real-world photographs [10] in the process of CG image generation, local artifacts are considered as evident forensics clues. Previous works have demonstrated that more texture features exist in shallow layers [40, 41]. Motivated by these findings, we first proposed the feature fusion module (FFM) to obtain local feature representations from shallow layers and then learn discriminative information through our proposed PCL.

In FFM, the shallow features from the first three downsampling blocks denoted as $$ (where layer = 0, 1, 2) are separately encoded into the latent space with the same spatial size as $$. These embeddings are concatenated together and further transformed into a low-dimensional embedding $$ through the patch projector (consisting of two 1 × 1 convolutional layers). Note that c is the spatial size of the local feature embedding, and h is the channel dimension. We denoted the vector at each spatial location in $$ and $$ as query embedding q_i and key embedding k_i, where i, j = 1,  2, …, c². In this way, each embedding represents the local feature corresponding to the respective spatial location.

3.4. ICL Module

The instance-wise contrastive loss effectively enhances the invariance between the spatial and frequency augmented views of a single input, simultaneously enlarging the difference between the augmented views of different categories. Consequently, the model can effectively learn more generalized knowledge compared with the CE-based models.

3.5. Overall Loss Function

4.1. Datasets

We followed the implementation details in [10]. All five datasets were separated into train, valid, and test datasets with a ratio of 7:1:2.

4.2. Experimental Settings

In our experiments, we followed the preprocessing strategy proposed in [10]. During training, we randomly cropped the images with a size of 224 × 224. We also used a class-balanced random sampler to balance the distribution of CG and PG images within each training batch. Each of the three spatial augmentations is conducted with a 50% probability. During evaluation, we centrally cropped the images with a size of 224 × 224 with no augmentation.

Our method is implemented with PyTorch library on two NVIDIA GTX 3090 GPU cards. EfficientNet-B4 (EN-B4) [20] pretrained on the ImageNet [46] is used as our backbone. The model is optimized by Adam with an initial learning rate of 1 × 10⁻³, a weight decay of 1 × 10⁻⁵, and a batchsize of 32. The total training epoch number is 30, and the learning rate decays by 0.4 every 5 epochs. According to the experience of the previous work [25], the hyperparameter α is set as 0.999, and the temperature parameter τ is set to 0.07. β is randomly selected from a uniform distribution between 0 and 1. The dimensions d and h are both set to 128, and the final feature map size c is set to 7. The number of negative samples K is set to 30,000. Following the previous work [10], we mainly use Accuracy (ACC) as the evaluation metric.

4.3. Intradataset Evaluation

We initially conducted a comparative analysis between our proposed FCL and state-of-the-art CG image detection algorithms within the same intradataset setting. The results are shown in Table 1, where the best results are highlighted in bold. On the whole, our FCL method demonstrates promising performance across the five datasets and achieves a certain improvement compared to other methods. Notably, when fairly evaluated on the recent dataset LSCGB, it exhibits a significant improvement from 91.45% to 95.11% compared to the best method (i.e., Bai et al. [10]). The excellent detection performance of FCL can be attributed to the designed data augmentation module and the comprehensive contrastive loss, which enables the detector to capture critical discrepancies between CG and PG images. On the other hand, we observed that the performance of our FCL is slightly lower than the second best method (i.e., Bai et al. [10]) in the case of Rahmouni et al. [7] and Columbia [42] datasets. However, in the following generalization and robustness evaluations, our FCL achieves much better performance. This demonstrates that the proposed method tends to learn intrinsic forensic features so that it does not easily overfit on small datasets.

4.4. Generalization Ability Experiments

The generalization ability of models is important in real-world practical applications. To demonstrate the generalization of our proposed FCL, we conducted cross-dataset evaluations. Specifically, the models are trained on the LSCGB and evaluated on the other four datasets, i.e., SPL2018 [44], Columbia [42], Rahmouni [7], and Tokuda [43].

The experimental results in terms of ACC (%) are illustrated in Table 2. Generally speaking, all the detectors experience performance degradation under cross-dataset evaluation. Compared with prior deep learning–based arts, our method has the least decrease in detection accuracy. Especially on the Rahmouni et al.’s [7] dataset, the second best method (i.e., Bai et al. [10]) experiences a decrease of 15.57%, while our method decreases only 7.26%. On the SPL2018 [44] and Columbia [42] datasets, the performance degradation of our method is 3.36% and 4.35%, respectively, which is smaller than other state-of-the-art methods. To evaluate the performance of the proposed model comprehensively, we adopted additional evaluation metrics including recall, area under the curve (AUC), and equal error rate (EER) and compared the performance with three state-of-the-art methods. The comparison results are obtained by training the models with their official public codes on our training samples. The results in terms of recall (%), AUC (%), and EER (%) are illustrated in Tables 3, 4, and 5, respectively. Our model achieved the best performance under all evaluation metrics. The proposed FCL outperforms Bai et al. [10] with average recall and AUC improvements of 8.68% and 2.87%, respectively. The average EER of FCL is 13.16% lower than that of Bai et al. [10]. This result shows that our proposed FCL framework takes advantage of more critical discriminate discrepancies between CG and PG images, which generally exist in different datasets. This mainly benefits from our designed framework which is proficient in learning a more general feature representation.

4.5. Robustness Experiments

In real-world scenarios, postprocessing operations such as compression and resizing may be used for the images transmitted over the Internet, which inevitably affect the performance of the detectors of CG images. Robustness is an essential indicator that reflects the ability of the detectors to resist these interferences in practical applications.

In this subsection, we evaluated the robustness of the proposed FCL against different postprocessing operations and compared our approach with current state-of-the-art methods. The models are all trained and tested on the LSCGB dataset. Following Bai’s experiential setting [10], we considered six image postprocessing operations commonly used in practical scenarios, including JPEG compression (quality factor (QF) = 70), scaling (upscaling 20% or downscaling 20%), median filtering (filter size = 3 × 3), mean filtering (filter size = 3 × 3), and Gaussian noise (zero mean and σ = 1). The testing results are shown in Table 6. Although all studied methods suffer a decline in performance when compared with results without postprocessing operations, our proposed FCL outperforms other state-of-the-art methods in most detection cases.

For instance, the accuracy of the proposed method decreases by 6.22% under median filtering, while the second-best method (i.e., Bai et al. [10]) suffers the decline of 17.58%. When encountering 3 × 3 mean filtering, our method drops 0.59%, while the second-best method (i.e., Zhang et al. [9]) drops 21.99%. This might be explained by the reason that these state-of-the-art methods tend to capture local-level forgery information, which can be readily destroyed by postprocessing operations.

Figure 3 shows the robustness evaluation against JPEG compression, downscale, and Gaussian noise with different parameters. The QF of JPEG compression is set to {90, 80, 70, 60}. The downscale ratio is set from 0.95 to 0.65. The Gaussian noise factor is set from 0.5 to 2. From Figure 3, it can be seen that as the interference intensity increases, the performance of our proposed FCL declines slowly when compared with other state-of-the-art methods. It indicates the superior robustness of our proposed model, especially under a high degree of postprocessing operations. This demonstrates that our proposed framework focuses on more essential distinctive features between CG and PG images. Such essential features are difficult to be affected by common postprocessing operations.

4.6. Ablation Experiments

In this subsection, we conducted ablation experiments to verify the effects of each component in our proposed FCL framework. The variants are all trained on the LSCGB dataset [10] and tested on the SPL2018 [44], Tokuda [43], and Columbia [42] datasets. Since generalization is the most important issue in the CG forensics task, we mainly focused on analyzing the generalization capability of different models.

5.1. T-SNE Feature Embedding Visualization

Firstly, we use T-SNE [49] to visualize the feature distribution of the output of our model. As shown in Figure 4, it can be observed that the output feature of the CE-loss–based model has a certain amount of overlap in the training dataset, even if they belong to different categories. Compared with the baseline, our model significantly reduces the overlap area between features of CG and PG images. The same phenomenon is observed on cross-dataset Tokuda, Pedrini, and Rocha [43]. The visualization of the reduced dimensional features further proves our model’s superiority in extracting intrinsic discriminative features.

5.2. Visualization for Detector’s Attention

We used the Grad-CAM to visualize the decision regions of the model. As shown in Figure 5, the heatmaps generated by our model highlight different regions for PG and CG images. The first three rows indicate the model’s attention on PG images, and the last three rows indicate the model’s attention on CG images. For PG images, compared with the baseline, our model pays more attention to local texture-complicated areas. For CG images, our model can locate more regions and contain more details where the artifacts exist. This illustrates that our method is efficient in capturing more comprehensive clues than the baseline method by focusing on more extensive areas.