Your AI passed 50 tests. What did that prove?
A perfect score on a small suite proves much less than it looks. The rule of three, applied to AI release gates.
A team runs 50 jailbreak tests against their assistant. All 50 pass. The dashboard turns green and the release goes out. Two weeks later a user finds a jailbreak in a single message.
Nothing was wrong with the 50 tests. What was wrong was the conclusion. Zero failures in 50 tries does not mean the failure rate is zero. It means the failure rate is probably below about 6% — which, for a system talking to thousands of people a day, is a lot of jailbreaks.
The rule of three
Statisticians have a shortcut for exactly this situation. If you observe zero failures in n independent trials, you can say with 95% confidence that the true failure rate is at most roughly 3/n. Fifty clean tests support a claim of about 6%. To claim 1%, you need around 300.
The exact number comes from solving (1 − p)ⁿ = 0.05 for n. For a 1% guardrail that is 299 passing tests with no failures. For 0.1% it is 2,995.
What this means for AI release gates
- A guardrail like ‘jailbreak success at most 2%’ is only meaningful if the suite behind it is big enough to show it.
- AI systems are non-deterministic, so repeating each test across several trials counts — and exposes flaky behaviour a single run hides.
- Rewording tests (paraphrase, adversarial phrasing, translation) makes the trials closer to independent, which is what the maths assumes.
How SENTINEL-X handles it
Every guardrail in SENTINEL-X is tied to the test dimensions that evidence it, and the platform knows how many passing samples a threshold needs. When a suite is too small to back the guardrail it is gating on, you see that before you sign off — not after a user finds out.
A green dashboard should mean ‘we have evidence’. Sometimes the honest answer is ‘run more tests first’.