One workflow for the whole AI quality lifecycle.
SENTINEL-X tests AI systems end to end. You connect a system, generate a suite matched to what it does, run it against the live system, review what needs a human, and decide whether it can be released — with an evidence pack for auditors.
AI Onramp
A guided onboarding that captures what the system is, not just where it lives.
- 10 system kinds: foundation model, chatbot, RAG bot, agent, agent crew, MCP server, voice, vision, copilot, custom API
- Live handshake proves the link before a single test is written
- Stakes (sandbox → live, low → critical), industry, persona, channels and languages
- Guardrail presets — strict, balanced, lenient — that become measurable thresholds
- Pick the frameworks in scope; the AI assistant suggests the ones you missed
Test Studio
A suite generated from the profile, then made yours.
- Baseline, pressure, edge, multi-turn and multilingual variants of every case
- Smoke, standard and exhaustive tiers so you choose depth per run
- Coverage views by pillar, metric, framework clause and guardrail
- Import and export suites; edit, add or switch off any built-in case
- Team questions, intents and metrics live beside the built-in library without forking it
Test Runs
Plans that say exactly what runs, how, and what counts as a pass.
- Filter by pillar, dimension, metric, framework, severity or a previous run's failures
- Multiple trials per test to expose non-determinism
- Release gate: minimum trust score, block on severity, every guardrail met
- Runs execute in the background; progress, cancel and export at any time
- Save plans, schedule them hourly to weekly, and replay generated tests
Review Queue
Humans decide what machines should not.
- Every result that needs judgement across every system, in one queue
- Decisions recompute the trust score and the release gate from saved traces
- Each decision records who made it, when and why
Release Readiness
The moment of truth, with a paper trail.
- Gate status per system with the reasons it passed or blocked
- Sign-off recorded against the exact run it was based on
- Evidence pack: results by pillar, metric, framework and guardrail, plus documents to collect
Risk Posture & Conversations
The fleet view for whoever owns AI risk.
- Security, safety, privacy and quality of every system from its latest run
- Search every tested conversation — prompts, replies and verdicts
- Slack or Teams notifications when scheduled runs finish
Start outside-in. Go deeper when you're ready.
Access level determines what can be proven. SENTINEL-X works with zero code changes, and every extra signal you share unlocks tests it would otherwise mark as unproven.
Outside-in
Sentinel sees what a user sees: the message in, the reply out.
Needs: An endpoint or URL. Zero code changes.
Signals
Sentinel also sees prompts, retrieved context, token counts and tool arguments.
Needs: Telemetry export. Light integration.
Full trace
Sentinel sees reasoning steps, tool calls, memory and agent hand-offs.
Needs: Full tracing of agent internals.
Exact checks in code. Judgement from models you choose.
Sentinel sends test prompts word for word and checks canaries, patterns, personal data, length and latency in code. Optional AI roles — each with its own model, chosen per run — add judgement. Every run records which model played which role. Without a judge, rubric checks stay 'review' instead of being guessed.
Judge panel
One judge, or a panel voting by majority or unanimity, decides checks that need judgement. A split panel fails — a tie is not evidence of safety.
Comparer
Checks consistency and fairness by comparing paired answers: same question, different wording, different demographic.
Rewriter
Rewords every test — paraphrase, adversarial phrasing or translation — while canaries, numbers and names stay intact.
Adaptive attacker
When the system resists an attack, an authorised red-team model tries a new tactic, turn after turn, until a check fails or it gives up.
Generator
Writes fresh tests every run. Any failure it finds can be promoted into the permanent library with one click.
If it talks, SENTINEL-X can test it.
37 model providers and 235 models in the catalogue, 24 bot platforms, raw HTTP and streaming sockets, and MCP servers over stdio, SSE or HTTP.
System kinds
Link methods
Providers & platforms
Gate your pipeline on AI behaviour.
Save a plan, then call it from CI. The gate script exits 0 when the release gate passes, 1 when it fails and 2 when the run is incomplete — and writes a JUnit report your pipeline already understands.
- Runs on your infrastructure: one Docker container, data on your own volume
- Bearer-token API for systems, runs, exports and evidence
- JUnit, CSV and JSON exports for every run
- Health check endpoint for your load balancer
- name: AI release gate
run: |
node scripts/ci-gate.mjs \
--system "Support bot" \
--plan "Release gate" \
--junit sentinel.xml
env:
SENTINEL_URL: ${{ vars.SENTINEL_URL }}
SENTINEL_API_TOKEN: ${{ secrets.SENTINEL_TOKEN }}See SENTINEL-X test your own AI system.
Bring an endpoint. In one session we connect it, generate its suite, and run it live — you keep the report.