A detector can score well by learning a distinction its designers never intended. To understand whether it has learned anything useful about image generation, start with the examples it was given—and the differences it was rewarded for finding.

Imagine building a training collection with camera photographs on one side and generated images on the other. The labels are correct. The model performs well on a random test split. Yet the photographs mostly carry JPEG compression, while the generated examples arrive through a different export pipeline.

The model has no obligation to prefer your intended explanation. If processing history predicts the label, it can become a useful shortcut during training. Success on the collection may then overstate success on the question you actually care about.

An image passes through a frozen encoder and a trainable head; labels supervise the head.
One research setup: the encoder stays fixed while labelled examples train the classification head. Schematic.

The label defines the lesson

Supervised learning begins with examples and target answers. A classifier receives image data, produces a prediction and incurs a loss based on the supplied label. Optimization adjusts its trainable parameters to reduce that loss over many examples.

Labels must have an operational meaning. Does “AI” mean fully generated? Does it include a photograph with a generatively replaced background? Are conventional edits allowed in the photographic class? A collection cannot answer these questions through file extensions alone.

Known creation records and carefully documented collection procedures are stronger starting points than labels inferred from another detector. Otherwise, a new model can end up learning to reproduce the older model's mistakes.

At ImageSignals, the existing pilot methodology treats origin records as ground truth and evaluates detector predictions against them. Its convenience sample has explicit limits; it is not a representative survey of internet images. That distinction should survive any editorial use of its results. [8]

Pixels become a representation

An RGB input with dimensions 224 by 224 contains 150,528 channel values. That arithmetic is useful for imagining the input, but 224 is an example—not a universal requirement. Some systems resize, some crop, some use multiple views, and some operate at other resolutions.

A neural network transforms its input into intermediate representations. An embedding is one such representation: a vector of values that can be used by a downstream decision rule. Its dimensions are not usually named human concepts. It is misleading to label one bar “fingers” and another “shadows” unless that interpretation has been established.

The second ImageSignals video follows a concrete research example: a pretrained CLIP encoder and a simple learned classification head. Ojha, Li and Lee investigated linear classification and nearest-neighbour approaches in that feature space, reporting improved transfer to their tested unseen generators. That is evidence for a useful representation in a specified evaluation, not proof of permanent universal detection. [1]

Learning the boundary can be simpler than learning the representation

For a linear classification head, the score takes the form:

score = w₁z₁ + w₂z₂ + … + wₙzₙ + b

Here, z is the representation, w contains learned weights and b is an offset. A higher score can be defined to favour the generated class. A loss function links that score to the target label.

In a frozen-encoder setup, training changes the head while leaving the encoder fixed. In end-to-end training, parameters inside the encoder can change too. These are different training choices; neither follows simply from calling the system a neural network.

The geometry shown in an explainer is an analogy. A two-dimensional plot is not automatically a measured projection of a model's embedding space, and coloured dots are not experimental predictions unless their values and provenance are supplied. The video's illustrative diagrams should retain that status when adapted to an article.

What counts as a useful feature?

There is no single feature family shared by all detectors. Research has examined spectral discrepancies, neighbouring-pixel relationships and representations learned through broader image tasks. Durall and colleagues studied spectral properties associated with particular upsampling pipelines. Tan and colleagues developed an approach based on local pixel relationships. Neither establishes a permanent signature of every synthetic image. [2, 3]

A frequency spectrum is a way to describe spatial variation in an image. Repeated structure may be visible there even when it is unobtrusive in the picture. But resizing and compression can also affect that structure. Seeing a peak does not, by itself, identify AI generation.

This is why an upsampling demonstration belongs beside a clear caption: it illustrates a processing effect. It does not reproduce a detector's internal attention, and it does not turn the sample photograph into a generated image.

When a feature becomes a shortcut

Grommelt and colleagues investigated compression and image-size biases in GenImage. Their preprint shows why unequal processing across classes can distort what the studied detectors learn and how their generalization is assessed. The claim concerns the tested data and models, rather than every detector. [4]

One reported comparison is worth keeping precise. In their Table 4, average cross-generator accuracy for ResNet50 changed from 71.68% to 82.74% between the classic and constrained training conditions. That is 11.06 percentage points, under their protocol. It is not a universal gain from converting a file format, nor an ImageSignals test.

PNG is losslessly compressed: it stores its pixel information without discarding it in the way a lossy codec does. The relevant distinction here is the absence of JPEG's lossy compression artifacts, not the absence of all compression. A file extension alone does not establish whether an image previously passed through JPEG encoding. [9]

Controlling compression and size helps examine specific confounds. It does not establish that every other shortcut has disappeared. The photographic and generated collections may still differ in subjects, source communities or visual style.

Better data means better comparisons

One research direction tries to align the content of the two classes more closely. B-Free constructs training examples using self-conditioned reconstructions and content augmentation, reducing some easy semantic differences. Its name describes a method; it is not a guarantee that a dataset contains no bias. [5]

Diversity matters as well. Community Forensics investigates training with a broad collection of generators and studies the effect of generator diversity while controlling the image budget in an experiment. Its findings support broader coverage as one route to better transfer. Thousands of checkpoints should not be described as thousands of unrelated architectures, and generator diversity cannot stand in for every acquisition condition. [6]

The key editorial question is therefore not simply “how many images?” It is “which differences can the model exploit, and which conditions remain untested?” More examples of the same narrow process may add less useful coverage than a carefully designed comparison.

A test split can be independent in name only

A random split can place near-duplicates, images from the same source, or outputs of closely related generators on both sides. That may test performance on familiar conditions while leaving transfer unresolved.

A more informative evaluation separates training from threshold selection and final testing. It can hold out complete generator families or photographic sources, control processing across classes, and compare an original with its transformed versions. The purpose is to make the intended claim match the test.

The original GenImage benchmark includes cross-generator and degraded-image tasks. These are distinct evaluations. Accuracy on familiar, clean files should not be substituted for performance after compression or on a later generator. [7]

This does not mean every new generator defeats every detector. It means that success must be demonstrated rather than inherited from a familiar benchmark.

What a good training story should tell you

Ask what the labels mean, where both classes came from, which parameters were trained, and which generators and photographic sources were held out. Ask how preprocessing differs between training and use. Finally, ask whether the reported number describes ranking, decisions at a threshold, or probability calibration.

No single answer settles model quality. Together, they reveal whether an impressive result addresses the image collection you need to examine.

Our methodology explains the scope of the ImageSignals pilot and how its results are interpreted. To understand what a score means after training, read How AI Image Detectors Actually Work, when you want to explore the result in more detail.

The training process rewards distinctions that help solve the supplied task. The research challenge is to make those distinctions survive beyond the collection that taught them.

Watch the video

How an AI Image Detector Learns the Difference · ImageSignals · 5:34.

Watch on YouTube.

This article draws on research developed for the ImageSignals video series and expands it for readers.

Sources & further reading

  1. Ojha, Li & Lee, Towards Universal Fake Image Detectors That Generalize Across Generative Models, CVPR 2023, approach and evaluation.
  2. Durall et al., Watch Your Up-Convolution, CVPR 2020.
  3. Tan et al., Rethinking the Up-Sampling Operations in CNN-based Generative Network for Generalizable Deepfake Detection, CVPR 2024.
  4. Grommelt et al., Fake or JPEG?, 2024 preprint, bias analysis and Table 4.
  5. Guillaro et al., A Bias-Free Training Paradigm for More General AI-generated Image Detection, CVPR 2025.
  6. Park & Owens, Community Forensics, CVPR 2025, especially §4.3.
  7. Zhu et al., GenImage, NeurIPS 2023, official benchmark documentation.
  8. ImageSignals, Methodology and transparency. No new training or benchmark was run for this article.
  9. W3C, PNG Specification, Third Edition, lossless format definition.