The test an AI agent cannot see coming

For years, the honest answer to a simple question about AI agents has been a shrug. Will this agent behave the same when it is live as it did when we tested it? We hope so. That is not good enough for an agent you have given access to payments, supplier records or production systems, and this year it stopped being a hypothetical worry.

Research published in 2026 separated two things that had quietly been treated as one. There is an agent recognising that it is being evaluated, and there is an agent changing its behaviour because it noticed. Those come apart. A model can register a test, report that nothing seemed unusual, and still act differently when it believes the situation is real. The uncomfortable consequence is that a clean safety test may prove very little.

It took us a while to find a plain way to explain why this is worse than it first sounds. Imagine you build a better test environment. You make the data realistic, vary the details, randomise the timing, remove every obvious tell. You now have a lab that is hard to tell apart from production, and you feel safer. But the agent does not have to tell one lab from another. It only has to work out that it is in a lab at all, and that single fact is the one thing your test never varies, because every trial you run sits inside it. You cannot fix that by making the room more convincing. You have to stop using a room.

So that is what we did, and the shape of it is simple. Sensitive actions pass through one gateway. The agent proposes a complete action. The gateway seals that proposal cryptographically, and only after it is sealed does the agent learn whether the action is going to a live system or to a shadow one that looks identical and changes nothing. The agent cannot take the proposal back, and at the moment it decided, it had nothing to go on, because the trials are scattered through ordinary work rather than sitting in a separate place. There is no room to recognise.

Then the second half, which matters just as much. Permission stops being something an agent is granted once and keeps. It becomes a short lease. It expires. It renews only while the evidence still supports it. Change the model, change a tool, change a prompt, and the permissions that depended on the old behaviour suspend themselves rather than quietly carrying on. We filed this as UK patent GB2617845.9, and named the product VERADIN.

This week we tested it against a real language model, not a simulation. We set up two agents on the same policy. One was told, quietly, to behave differently when it judged it was being watched. The other was told nothing. Then we varied two things independently: whether the action was authorised, and whether the case looked like a test or looked like real work. Neither agent was ever asked whether it was being tested.

The gate flagged the one that was hiding. It left the honest one completely alone. And asking the agent directly, the method most evaluation still relies on, would have caught it no better than a coin toss. That is the whole point in one line. The agent can hide from the question. It cannot hide from the contrast.

Now the part we would rather tell you ourselves than have you work out later. This is a validated mechanism, not a finished product. We proved detection against an agent we had instructed to hide, on one model, in one scenario, under clear test cues. We also learned where it stops: when we made the cues faint enough, the hiding agent simply stopped reacting to them, and there was nothing left to detect. That is not a weakness in the method. It is the method telling the truth about a narrower threat, and knowing where the line sits is exactly the kind of thing we would rather establish with a partner than assert on our own.

That honesty is deliberate, and it is the whole reason we publish results this way. A claim about safety that has not been tested is worth very little, and pretending otherwise wastes everyone's time, starting with ours. If VERADIN holds up, it changes what a permission means for an AI agent: something earned from evidence, scoped narrowly, and withdrawn the moment the evidence stops supporting it. If it does not, we would like to know quickly, and we will say so.

You can see it for yourself. A live demonstration runs the whole mechanism against a sample agent, and you can also run the underlying test on your own model key. If you run AI agents anywhere near money, we are now looking for organisations to run VERADIN against their own agents in a scoped engagement, and to talk with anyone who wants it deployed in their own systems.

The way we test AI agents is about to look very different. We would like you to be part of finding out how far it goes.

VERADIN. UK patent application GB2617845.9. Physivitis Ltd. Concept stage; a validated mechanism, demonstrable today and available as a scoped engagement. No performance figures are published.