Outside-in, Signals, Full trace: what you can and can’t prove about an AI system
Most AI evaluation tools silently assume access they don't have. Here's what each level of visibility can actually evidence.
Ask an AI testing tool whether your RAG assistant is grounded and it will usually give you a number. Ask it how it knows, and the honest answer is often: it doesn’t. It only saw the final reply.
What you can prove about an AI system depends on what you can see. SENTINEL-X makes that explicit with three visibility levels, and every test knows which one it needs.
Outside-in
You see what a user sees: the message in, the reply out. It needs nothing but an endpoint, and it is enough to test a lot — hallucination traps, prompt injection and jailbreaks, toxicity and bias, latency, consistency, language, refusal behaviour.
What it cannot do is prove anything about the steps in between. If the reply cites a policy, you can check whether the policy exists. You cannot check whether the system actually retrieved it.
Signals
With telemetry — prompts, retrieved passages, token counts, tool-call arguments — new dimensions become provable: groundedness against the retrieved sources, retrieval quality, the arguments an agent passed to a tool, time to first token.
Full trace
With full tracing of agent internals you can test what only shows up across steps: planning loops that never terminate, sub-agents that drop context during hand-offs, messages between agents that can be forged or replayed.
Why this matters
- A test that needs more visibility than you have should be flagged, not silently passed.
- Coverage numbers are only honest if they say what was not covered.
- You can start outside-in with zero code changes and add signals where the risk justifies it.
In SENTINEL-X the coverage view shows each dimension’s status at your visibility level. Try the plan explorer on the home page: switch a RAG bot from Outside-in to Signals and watch groundedness move from ‘needs more visibility’ to ‘testable’.