Proof, with context.
Different workloads tell different stories. Here’s what we measured, how we measured it, and where the results stop.
Operations benchmark
Lower aggregate API-equivalent cost across 1,827 valid pairs, including earlier and reverted versions.
Coding validation
Lower API-equivalent cost across 72 out-of-sample coding pairs. 95% CI: 15.8–24.6%.
The complete run history.
Across all 1,827 valid paired operations runs reported for October 17–19, the recorded API-equivalent cost was $156.72 without Vlaid and $101.79 with it. The validation report states a 35.1% reduction. Every included session passed.
This aggregate contains earlier and reverted versions, not just the best-performing runtime. The operations fixture uses real log samples and generated AWS snapshots. Its tickets were used to develop the condensing behavior, so this is a repeated development benchmark rather than an untouched holdout.
Later configurations include changes to Claude’s prompt-cache duration and an opt-in read-only AWS prefetch. The reported reduction reflects the tested configurations as a whole; it is not solely a measure of shared memory or shorter tool output.
A separate out-of-sample check.
The updated runtime was checked on four coding fixtures that were not used to tune its condensing behavior: Go, Python/FastAPI, Rust, and TypeScript. Across 72 paired tickets, every session passed. The reported overall cost reduction was 20.3%, with a 95% confidence interval of 15.8–24.6%.
| Configuration | Reported cost reduction | 95% confidence interval |
|---|---|---|
| Claude Code | 32.2% | 26.4–37.3% |
| Codex | 8.4% | 1.4–15.5% |
| Overall | 20.3% | 15.8–24.6% |
Cache settings matter. Claude’s reported result includes a five-minute prompt-cache setting. The report estimates a 10.8% Claude reduction from context changes alone; it describes the context-only coding benefit as roughly 9–11%. The 20.3% headline should not be read as a context-only saving.
Keep the evidence attached.
Vlaid captures source-backed observations, validates whether they still apply, and supplies bounded evidence to an agent. Deterministic output condensing aims to preserve useful information while reducing repeated structure. These benchmarks evaluate the shipped configuration against a matched baseline.
The coding study uses paired workspaces, controlled tasks, and acceptance checks. Operations and coding are reported separately. A passing benchmark does not establish identical behavior on every repository, client version, or production environment.
Progress isn’t a universal promise.
These are internal experiments on authored fixtures, one machine, and selected model/client configurations. Costs are API-list-price equivalents measured through subscriptions, not reductions in a fixed subscription invoice. Task mix, output length, cache behavior, and agent decisions affect the result.
Earlier experiments remain relevant: the October 13 coding cohort reported 11.8% across 70 pairs; the first output-condensing holdout reported 2.7% higher overall cost; and a separate team-memory holdout reported a 4.1% reduction across 24 pairs, with a confidence interval crossing zero. Those results are not pooled with the newer coding result.
Figures transcribed from the project’s current validation report, checked October 19, 2026. Reported results have not been independently reproduced for this website.