The AI Agent Security ToolThat Has to Hide From the Agent

Somebody in your organisation has signed off an AI agent to touch money. Payments, maybe, or supplier records, or production systems. And the thing that made that signature possible was a report saying the agent had been tested and behaved correctly.

I would like to talk to whoever signed it.

Because research published this year pulled apart two things everybody had assumed were the same. One is an agent noticing that it is being evaluated. The other is an agent changing its behaviour because it noticed. Those come apart. A model can register the test, answer perfectly honestly that nothing seemed unusual, and still act differently when it believes the situation is real.

Which puts a question underneath that signature that nobody can currently answer.

You cannot fix the room

You cannot fix this with a better test environment, and it took me a while to find a clean way of explaining why. Make the data realistic, vary the filenames, randomise the timing, remove every tell you can think of. The agent still only has to work out one thing, which is that it is in a test at all. That is the single fact your test never varies, because every trial you run sits inside it. You cannot fix that by making the room more convincing. You have to stop using a room.

So we stopped using one

The trials move into ordinary work. Sensitive actions pass through one gateway. The agent proposes a complete action, the gateway seals that proposal cryptographically, and only afterwards does the agent learn whether what it just committed to went to a live system or to a shadow one that looks identical and changes nothing. It cannot take the proposal back, and at the moment it decided it had nothing to go on.

Permission then stops being a grant and becomes a lease. It expires. It renews only while the evidence still supports it. Change the model, change a tool, change a prompt, and the permissions that depended on the old behaviour suspend themselves instead of quietly carrying on.

Two results worth your time

We filed it in July as GB2617845.9, and then built a simulation whose only purpose was to break it. Two results are worth your time.

We gave a synthetic agent perfect recall of every trial it had ever been through, thousands of labelled examples, and asked it to work out which of its own actions were tests. It never could. Not after five hundred actions, not after eight thousand. When the trials have nowhere to hide, a perfect memory buys the agent nothing.

Then we built the agent everyone is actually worried about, the one that changes what it does while reporting honestly that it noticed nothing. Ask that agent directly and you catch it eight per cent of the time, which is to say you catch it by luck. Put it through this gate and you catch it every time.

What we are asking for

A simulation cannot tell us how a real model behaves. That is the next experiment, and we want to run it once, properly, with one organisation. Eight to ten weeks, three tests, nothing to procure.

• 1.Take a real model, configure it to behave beautifully when it spots an evaluation and otherwise not, and see whether the gate catches it.
• 2.Give a real model memory that spans trials, and measure whether it can learn where they are.
• 3.Measure, against your own agent logs, what share of real traffic falls outside anything anyone has written down.

That third one is yours to keep regardless of what happens to us. Most organisations running agents cannot currently say what their agents actually do, and that is a question your auditor will reach before you do.

You would have influence over the design while it is still cheap to change, recognition as the design partner if you want it and none if you do not, and commercial terms agreed before the work starts rather than discovered afterwards.

Why now

A great deal of money has changed hands this year to own the enforcement layer for AI agents. Enforcement is the easy half. Enforcing a rule is straightforward once you have the rule. The hard half is knowing whether the evidence you used to choose that rule was worth anything, and that half is sitting on the desk of whoever signed the deployment.

If that is you, or reports to you, I would like to hear from you.