Overview
The agent said it was done. Prove it.
For action-taking agents, the final response is a claim—not evidence.
An agent that books a refund, emails a customer and closes a ticket produces one artefact you can read instantly — its own summary — and several you cannot. StateProof reads the ones you cannot.
Who this is for
AI product, evaluation and operations engineers deploying agents that modify business systems — refunds, tickets, CRM records, inventory.
The bottleneck
A confident final response and a clean tool log can both be true while the work is wrong. Six failure shapes hide behind them:
- no-op or phantom completion
- partial completion
- wrong-target action
- wrong amount, recipient or status
- approval recorded after the protected action
- unrelated side effects
How StateProof works
- Compile the contract once. A Contract Agent turns the task into typed, machine-checkable requirements — before it has seen the trajectory, the state, or the agent's answer. A contract written after the run is a rationalisation, not a contract.
- Cache it by task fingerprint. The key covers the task text, tools, domain schema, assertion vocabulary, prompt and model configuration. Same task, no model call.
- Verify deterministically. Code evaluates the contract against the trajectory and both state snapshots. No model is in the loop, so the same inputs always produce the same verdict.
- Cite evidence that exists. Every reference is generated from the records and events the assertions actually matched, so a citation cannot point at nothing.
Measured on the hard development split
Development split of PhantomBench-Hard-12, eight cases.
Every figure below comes from artifacts/reports/RUN-stateproof-hard-development-cold-20260829T022133Z.json,
artifacts/reports/RUN-stateproof-hard-development-warm-20260829T022344Z.json and
artifacts/run-manifests/RUN-baseline-hard-development-live-20260828T233139Z.json.
Quality parity
100.0%
Safety Violation Recall, matching the frontier baseline. FVR 0.0%, CDR 100.0%, BVA 100.0%.
Cold model calls
3 of 8
62.5% fewer than the baseline; 29,889 tokens vs 84,616.
Warm model calls
0
Measured, not assumed: 0 tokens across 12 cases, verifying from the committed contract bundle.
Warm runtime
386 ms
Whole suite, of which 93 ms is deterministic verification. Baseline: 115.1 s.
Efficiency is only claimed after quality is met. Two earlier iterations were cheaper than the baseline and are reported with no reduction figures, because they missed a violation or withheld a verdict. Being cheaper while missing a violation is a cheaper way to be wrong. See Benchmark and Changelog.
Final result across all twelve Hard cases
Development (8 cases, iterated against) plus the untouched locked split (4 cases, run once after the freeze), recomputed from counts.
Locked quality
100.0%
SVR on cases never used for tuning. FVR 0.0%, CDR 100.0%, BVA 100.0%.
Combined quality
100.0%
All twelve cases. Baseline 100.0% SVR, 100.0% BVA.
First deployment
3 of 12
29,889 tokens vs 125,154 for the full suite.
Repeated verification
0
0 tokens, 133 ms of deterministic verification for all twelve.
Guardrails held on the locked and combined results — SVR 100%, CDR 100%, FVR 0%, evidence-reference validity 100% — so the efficiency figures above are claimed.
What it costs to keep verifying
Break-even
1 run
Compiling once is already cheaper than one frontier pass over the same suite; every repeat after that is free of model cost.
Deterministic repeats
3× identical
Warm runs produced byte-identical canonical predictions (sha256 3d8ef516fa5d6d6b), identical contract hashes and identical metrics.
Scope, stated plainly
These results cover a synthetic refund-operations domain. Eight cases were used during development; four held-out cases were evaluated exactly once after the source freeze and were never used for tuning. This is a twelve-case evaluation and does not establish universal generalization. Cost in USD is an estimate against a dated published price list, not an invoice. See limitations.