feat(web): the Evidence Matrix, read rather than computed (#708 view 2) - #747
Merged
Conversation
The last of #708's four views, and the one that waited for an engine record instead of being built on a guess. WHY IT IS A READ Building the matrix on request costs 11.9 / 12.9 / 14.3 seconds per session on the real ledger -- ~39 s of CPU for three sessions, on a page the console re-polls every 15 seconds. And over the ledger as a whole it does not run at all: `build_matrix` requires synchronous columns, so a PBO is only ever defined WITHIN a session whose trials share a bar count, and a page cannot pick that scope without inventing an operator's decision. So #726 made `trials pbo` record every field of its `PBOResult`, and this reads them. The console displays results an operator ran and never runs one on their behalf -- pinned by a test that makes `build_matrix` raise if it is called. AN UNRUN MATRIX IS NOT AN EMPTY ONE Three states, and the middle one is why the report carries `candidate_sessions` at all: no ledger, a ledger with columns and no recorded run, and a ledger with runs. The middle one is the state this ships in, and it is the only one that can be acted on. THE GUIDANCE NAMES A SESSION THAT ACTUALLY HAS COLUMNS `keel trials pbo --session all` is the obvious thing to print and would filter to a session literally called "all", find nothing, and print a refusal -- teaching an operator that the page does not know what it is talking about. So the report counts the trials `build_matrix` would ACCEPT (a per-bar series, not `series_missing`) and names one of those. On the tracked ledger that is `pbo-grid-entry-lookback-2026-07-20`, one of three genuine candidates. It deliberately does not check SYNCHRONICITY, which would mean reading every series -- most of the cost this module exists to avoid. Suggesting a session that might turn out ragged, and getting a clear refusal from the command, is a much smaller harm than a page that costs twelve seconds to render. PBO CARRIES NO JUDGEMENT It is the one figure a reader will want graded and grading it is exactly what the rail refuses: a high PBO beside a flat, positive OOS scatter is the GOOD outcome -- a broad plateau of near-identical configurations produces high PBO by construction. `trials pbo`'s own closing sentence says to read it alongside the degradation slope, never alone, and both cross plainly so a reader can. The dominance flags DO carry a state, because they are already verdicts, and they are three-valued: `False` says the distribution did not dominate, which is not what an absent field says. ⛔ No sortable column on the route and no sort key in the view. A matrix ordered by PBO is a leaderboard of overfitting scores, and `cscv.py` forbids PBO as a ranking key in its own source. TWO TESTS THAT WERE NOT TESTING ANYTHING `test_a_recorded_run_stops_telling_the_operator_to_run_one` seeded a ledger with no columns, so `suggested_session` was empty and the invocation was empty whatever the rule said -- it passed against a page that prints the command forever. And the bool-is-an-int guard was pinned through a fixture no writer can produce, so it asserted nothing; it is checked at the reader now, which is the honest way to pin a defensive check. Both found by mutation. So was a stale `__pycache__`: the mutant `MIN_COLUMNS_FOR_A_RUN = 2` -> `1` is the same byte length as the original, so restoring the source within one mtime tick left Python reusing the compiled mutant. Worth knowing for any same-length mutation. Eleven mutants killed: the matrix built on request, pre-#726 gauntlet rows read as matrices, a dominance flag read as a column count, an absent flag read as a denial, a backfilled session suggested, a one-column session suggested, the command still printed after a run, PBO gaining a judgement, the section dropped from the view, the table gaining a sort control, and the route becoming sortable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KZZxmspQXe5qJ9FAsG13s6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #708. Closes the honest cockpit milestone.
The last of #708's four views, and the one that waited for an engine record instead of being built on a guess.
Why it is a read
Building the matrix on request costs 11.9 / 12.9 / 14.3 seconds per session on the real ledger — ~39 s of CPU for three sessions, on a page the console re-polls every 15 s. And over the ledger as a whole it doesn't run at all:
build_matrixrequires synchronous columns, so a PBO is only ever defined within a session whose trials share a bar count, and a page can't pick that scope without inventing an operator's decision.So #726 made
trials pborecord every field of itsPBOResult, and this reads them. The console displays results an operator ran and never runs one on their behalf — pinned by a test that makesbuild_matrixraise if it's called.An unrun matrix is not an empty one
Three states, which is why the report carries
candidate_sessionsat all: no ledger · a ledger with columns and no recorded run · a ledger with runs. The middle one is the state this ships in, and the only one that can be acted on.The guidance names a session that actually has columns.
keel trials pbo --session allis the obvious thing to print and would filter to a session literally called "all", find nothing, and refuse — teaching an operator that the page doesn't know what it's talking about. So the report counts the trialsbuild_matrixwould accept and names one of those. On the tracked ledger that'spbo-grid-entry-lookback-2026-07-20, one of three genuine candidates.It deliberately doesn't check synchronicity, which would mean reading every series — most of the cost this module exists to avoid. Suggesting a session that might turn out ragged, and getting a clear refusal from the command, is a much smaller harm than a page that costs twelve seconds to render.
PBO carries no judgement
It's the one figure a reader will want graded, and grading it is what the rail refuses: a high PBO beside a flat, positive OOS scatter is the good outcome — a broad plateau of near-identical configurations produces high PBO by construction.
trials pbo's own closing sentence says to read it alongside the degradation slope, never alone, and both cross plainly so a reader can.The dominance flags do carry a state, because they're already verdicts — and they're three-valued:
Falsesays the distribution did not dominate, which is not what an absent field says.⛔ No sortable column on the route and no sort key in the view. A matrix ordered by PBO is a leaderboard of overfitting scores, and
cscv.pyforbids PBO as a ranking key in its own source.Two tests that weren't testing anything
test_a_recorded_run_stops_telling_the_operator_to_run_oneseeded a ledger with no columns, sosuggested_sessionwas empty and the invocation was empty whatever the rule said — it passed against a page that prints the command forever. And the bool-is-an-int guard was pinned through a fixture no writer can produce; it's checked at the reader now.Both found by mutation. So was a stale
__pycache__: the mutantMIN_COLUMNS_FOR_A_RUN = 2→1is the same byte length, so restoring the source within one mtime tick left Python reusing the compiled mutant. Worth knowing for any same-length mutation.Gates
E501is pre-existing indocs/experiments/)🤖 Generated with Claude Code
https://claude.ai/code/session_01KZZxmspQXe5qJ9FAsG13s6