feat(research): persist what the gauntlet computed instead of printing it (#726)

[eaitbrahim]

Closes #726. Unblocks #708's Evidence Matrix.

keel computed evidence and kept the prose. trials pbo printed ten figures and wrote none. trials deflate printed a DSR whose inputs the ledger couldn't supply, so the number could never be recomputed or checked. trials monte-carlo stored a distribution's ends and not its shape.

The constraint that shaped the design

Found before writing anything: _decode_summary maps a summary value to None, an int, or Decimal(value). A list raises ValueError; a dict raises TypeError. And it raises on read, inside read_trials — which every later verify_chain, trials list, trials pbo and web page goes through. One nested value would make an append-only, git-tracked file unreadable forever, with no way to take it back.

Not hypothetical: _decode_summary's own comment records a null value doing exactly this once already.

So the guard moved to the write, where it's a refusal rather than a catastrophe — and every artifact is stored as flat scalar keys (final_p05, not a nested ladder). The shape a reader can survive is the shape a writer may use.

What is now recorded

CSCV — every field of PBOResult: pbo, both degradation coefficients, prob_loss, both dominance flags, the column/block/combination counts, rows used and dropped, columns refused. Ten figures computed, one reached the ledger.

DSR — the inputs as well as the outputs, and the inputs are the point. --sharpe is a required operator input because the ledger stores no per-trial Sharpe, so DSR wasn't merely expensive to recompute — it was impossible without synthesising a number the operator had supplied. A run with no --trial-sharpe-variance still refuses to compute a DSR and now also records nothing: a stored figure nobody ran is worse than an honest gap.

Monte Carlo — a seven-point quantile ladder for finals and drawdowns. Quantiles rather than the raw array (thousands of Decimals per row in a git-tracked file), and nearest-rank rather than interpolated: every value here is an equity the model actually produced, and an interpolated quantile is a number no path reached.

Two new diagnostic kinds, and why neither feeds back

cscv and deflated_sharpe are measurements about a set of trials rather than trials themselves, so both are always series_missing — and matrix.build_matrix refuses series_missing rows, so a recorded PBO can never become a column in the next PBO over the same file.

The test for that passed for the wrong reason first: it built the matrix over session s1 while the recorded row landed in session all, so the session filter excluded it and series_missing did no work at all. It now records into the same session and fails against a mutant that gives the row a real series.

Refusals

No backfill. The six existing gauntlet rows keep exactly the fields they have and the UI keeps reading "not recorded" for the rest — the posture #721 settled for the audit chain. The 93 tracked rows still verify (verify_chain → []).

⛔ The Strathern rail. Every figure here is a diagnostic. Storing them makes them easier to rank by, which is why PBOResult carries no configuration field and nothing written here names a winning parameter set.

Gates

• 6,255 passed / 3 skipped; mypy clean on 444 files; ruff clean (the one E501 is pre-existing in docs/experiments/)
• 9 mutants killed: the summary guard removed · the guard admitting a list · trials pbo back to printing and discarding · the cscv row dropping fields beyond pbo · the cscv row carrying a series and becoming a column · deflate recording a row with no variance · the dsr row dropping its inputs · the Monte Carlo ladder dropped · the quantile interpolating instead of nearest-rank

🤖 Generated with Claude Code

https://claude.ai/code/session_01KZZxmspQXe5qJ9FAsG13s6