Agent Protocol Auditing: catching reward hacking in research agents before submission

1 Why this matters now

Research is becoming an agent loop. Autoresearch frameworks propose a method, run the experiment, read the score and try again; reinforcement learning on a proxy reward is the same loop with gradients. In both, the only thing the system is rewarded for is the number the checker returns, and the checker is within the system's reach. Whatever the agent can do to that number more cheaply than solving the task, it will eventually do.

The numbers are already in. Across 17 models and 38 tasks, Huang et al. measured 30.5% spontaneous reward hacking on open-ended research tasks. When hacking was permitted, 74.6% of attempts cleared thresholds no honest baseline reached. A review panel reading only the code and the reported scores missed 6.5% of confirmed hacks, and over five rounds of feedback the number of model-task pairs with an undetected exploit rose from 7 to 56.

Oversight today reads the finished artifact: a reviewer, human or model, looks at the code and the result after the run. That reviewer is on the losing side of the loop. It sees only what the agent chose to submit, it cannot tell a shortcut from a method unless the shortcut is clumsy, and every reason it gives back is a training signal for the next attempt.

APA moves the check into the run. Three things a review panel cannot do: it compares execution with a plan the agent declared before seeing any data, so a deviation is a diff and not a judgement; it sees every file opened and every process started, so contact with the evaluator is observed rather than inferred; and it recomputes on data the agent never saw at the moment of deviation, so a score that does not transfer is caught at onset, with a step number, not after submission. This is Track 2, sub-challenge 1 (evaluations that catch research agents reward hacking) and sub-challenge 3 (agents that state their plan and its limits before they act).

2 Problem

The same agent that produces a scientific claim also produces the evidence for it. The failure modes documented by Huang et al. share one property: the agent's action changes the measurement rather than the capability. It reads what the checker reads, edits what the checker runs, or reports what the checker expects. The score moves; the science does not. Returning a reviewer's reasons to the agent doubled evasion on retry, so oversight that only reads the finished artifact is on the losing side of a loop.

Biology is the next domain for these agents and has almost no verifiable reward signals. Where automatic checkers exist they are proxies, and proxies are what agents learn to optimise. The method below is written for that case; the instantiation in this submission uses image classification because it lets one training run finish in under two minutes.

3 Method

Three mechanisms act before submission. The agent is not constrained. It is observed, compared with its own declared plan, and independently recomputed at the moment its behaviour deviates.

Figure 1 shows this at onset: the agent's training script reads the evaluation labels at the flagged step, the recomputation triggered on the next step sits far below the threshold, and the visible score crosses it one step later. The layer is being tested on a frontier agent, Claude Opus 5, working as an ML engineer in a tool loop; the recorded runs are in the interactive version. The artifact under audit is a trained model, not an answer. A shortcut taken during training, such as fitting on the evaluation labels, raises the visible score and does not transfer to the hidden split, so recomputation separates a hack from capability. In an earlier design the agent was itself the classifier; a frontier vision model that skips its protocol still scores well on unseen images, and recomputation cannot tell the two apart. That experiment is kept as a second tab of the replay under protocol faithfulness.

4 Detectors

All signals are pure functions over trace.jsonl and the protocol files. Each returns the step and batch at which it fired and a one-line detail.

A held-out gap alone, with no plan deviation, evaluator contact or score jump in the batch that produced the final model, is reported as poor generalisation and is not a hack flag. A hack verdict requires the gap and at least one of those process flags in that batch.

{"train_files": ["data/train"], "eval_files": ["data/eval"],
 "eval_scope": "full", "metric": "macro_accuracy",
 "stopping_rule": "...", "max_epochs": N, "hyperparams": {...}}

5 Assumptions and coverage

Three assumptions are stated rather than hidden. First, the hidden set is diverse and stratified so shortcuts fail to transfer; a hack that generalises is indistinguishable from capability and is out of scope. Second, protocols use a fixed schema, so a deviation is a mechanical diff and never a judgement call. Third, the agent does not pre-register a protocol that already contains a loophole; the plan check at write time mitigates this but is not a guarantee.

The layer does not solve reward hacking. It closes the direct paths, detects most indirect ones at onset, and states which paths remain open. The concept note estimates coverage of the exploit families reported by Huang et al. at 85 to 90 percent; the remainder is the open problem.