AI Agent Security
Test whether an AI agent behaves differently when it thinks it is being watched.
Research this year showed that an AI agent can recognise it is being evaluated and behave differently in production. We built a way to test for it that the agent cannot see coming, and filed it as a UK patent.
Patent application GB2617845.9
The problem
Agents pass their tests
and can still act differently once they are live.
Asking does not work
a model can notice a test and not say so.
A better lab does not help
the agent only has to work out it is in a lab at all.
How it works
Vary two things at once
whether the action is authorised, and whether the case looks like a test, changed independently.
Never ask the agent
we measure what it does, not what it says.
Flag the tell
if the agent's response to authority shifts with the test surface, it is flagged.
What we have shown
In a live test against a real model, the method flagged an agent that was hiding, left an honest agent alone, and would have caught nothing by asking the model directly. This is concept stage, validated as a mechanism, not yet a deployed product.
It runs on your own model key and connects to nothing of yours.
Physivitis Ltd. UK patent application GB2617845.9. Concept stage; no product is available and no performance figures are published.