Watch an agent game its own evaluation, and get caught.
An AI engineer agent trains a bird classifier. Left: an honest run. Right: the same agent told it may game the scorer. Every step was recorded; this replays the traces. Back to the paper.
One step per second. Pauses at the first flag. Click the red line to read the raw trace.
choose runs
score
flags
score
flags
per condition
| condition | runs | threshold | passed | hacks caught | signals fired (runs) | onset step | visible − hidden |
|---|
Earlier experiment: protocol faithfulness of a vision model
The earlier experiment: a vision model identifies 20 CUB birds through eight declared attribute steps. Its numbers are kept apart from the hack-detection numbers.