[zzylol]

Closes #1006. Part of #1008.

Why

For the ratio-of-quantiles workload in #1006, the Pass 1 memo stands for many complete candidate DAGs (49 = 7 × 7 in production, where the two equal quantiles share a group; #1006's 343 was measured without series identity; see #1024). The viewer's Stage 3 list shows only the selected plan and the rejections Stage 3 recorded, so nobody can see how many DAGs the memo stands for, how each differs from the selected plan, or whether any is cheaper than it.

What

A new Enumerated DAGs panel in tools/dag-viewer, between the Stage 0 and Stage 1 lanes. It reads memo.enumeration (written by #1003, priced by #1004) and is hidden for documents without it.

• Header: the DAG count after each memo step (Stage 0: 1 · Pass 1: 49 · Pass 2: ≥ 1000 · concrete states: ≥ 1000; ≥ and the limit when counting stopped), how many of the enumerated step's DAGs the document lists, and, when priced, how many are cheaper than the selected plan.
• One row per listed choice: the groups whose candidate differs from selected_choice (both choices are sparse: a group a choice leaves out takes candidate 0; a selected group the DAG does not read shows as G9 not read) (G7 → #2 CountSketch+heap (pass1): group, candidate index, and the candidate's label and origin from memo.steps when that step lists it, else just the index); budget share < 1 when budget_share_1 is false; ✓ selected; and, when priced, the cost and validity (✗ invalid with the rejection cause). A row with cheaper_than_selected is red.
• Clicking a row draws the choice's dag in the Stage 1 lane with the same renderer as a Stage 1 candidate (render(1, dag, false, null)). The page composes nothing. The Stage 2 lane is cleared, since the document has no physical plan for an enumerated DAG. compose writes one root per root group in increasing group id; the panel maps them to queries through the step's roots_of, so the lane has one root per query (a group answering two queries is both their root), or labels them root group #n when the step's memo is absent.
• When the enumerated step is earlier than the last step (selected_choice is over the last), the header says so, only each row's own groups are compared, and no row is marked selected.
• A choice that does not compose ("dag": null with error) shows its error and can't be clicked.
• The viewer refuses a document whose memo.enumeration is malformed, as it does for other sections.

How

All new logic is in enumeration.js: the checks and the row view are DOM-free; renderEnumeration fills the panel and handles clicks with one delegated listener. app.js gets three hooks: it adds validateMemoEnumeration to the document check, calls renderEnumeration(doc, showEnumerated) after loading a document, and calls pressEnumeratedRow(null) in selectLogical/selectPhysical, so that picking a Stage 1/2 candidate releases the enumerated row. showEnumerated is a new 10-line function in app.js that draws the DAG in lane 1.

Key code interfaces added / modified

tools/dag-viewer/enumeration.js (new)

/// Every shape problem of doc.memo.enumeration as strings; [] without memo or enumeration.
/// Called by app.js's document check next to validateStagePipeline.
function validateMemoEnumeration(doc); // new
/// The panel's content, or null without memo.enumeration:
/// { counts, listing, priced, against, rows: [{ id, changes: [{group, index, origin, label}], text,
///   nodes, budgetShare1, selected, priced, valid, rejection, cost, cheaper }], rootQueries }.
function enumerationView(doc); // new
/// dag with one root per workload query (query q's root is the root of the group whose roots_of holds q);
/// null when rootQueries does not cover every query.
function enumeratedDagPerQuery(dag, rootQueries, queryCount); // new
/// Fills #enumPanel for doc; a row click calls draw(id, dag, description, perQueryDag).
function renderEnumeration(doc, draw); // new
/// Marks row id (null: none) as the one the Stage 1 lane draws.
function pressEnumeratedRow(id); // new

tools/dag-viewer/app.js

/// Draws enumerated DAG id in lane 1 and clears lane 2; the draw callback renderEnumeration gets.
function showEnumerated(id, dag, description, perQueryDag); // new (inside the page's closure)
/// Draws dag in lane i; ids names its roots' queries in order (default: the workload's query ids).
function render(i, dag, physical, costPerNode, ids = queryIds); // was render(i, dag, physical, costPerNode)

The document format is unchanged by this PR: it reads memo.enumeration as #1003/#1004 define it (step, per_step, limit, choices[] with id, choice, dag, budget_share_1, selected and optional valid, rejection, cost_per_second, cheaper_than_selected; priced), plus memo.selected_choice and memo.steps[].groups[].candidates[] for origins and labels.

Usage on #1003's committed example: open the Ratio of quantiles (memo after each step) tab (python3 tools/dag-viewer/server.py, then http://localhost:8000/#memo-ratio-of-quantiles). The header reads Stage 0: 1 · Pass 1: 49 · Pass 2: ≥ 1000 · concrete states: ≥ 1000 · listed: all 49 Pass 1 DAGs · each row: groups whose candidate differs from the selected plan’s choice, which is over the concrete states memo: …. Row E2 reads G4 → #1 Project ← SummaryEstimate Quantile(q=0.9) ← SummaryAgg Kll(failure_probability=0.001 k=345) (pass1) · G6 → #0 …. Clicking it draws its DAG in the Stage 1 lane, with its root as q1's root.

The same with the priced hand-written fixture:

# Example 1's document plus a hand-written priced memo (4 Pass 1 DAGs: E2 selected, E3 cheaper, E4 invalid)
mkdir -p tools/dag-viewer/out && python3 -c "import json; d=json.load(open('tools/dag-viewer/examples/planner-layering-example1.json')); d.update(json.load(open('tools/dag-viewer/test-fixtures/memo-enumeration-small.json'))); json.dump(d, open('tools/dag-viewer/out/enum.json','w'))"
python3 tools/dag-viewer/server.py   # then open http://localhost:8000/?doc=out/enum.json

The panel header reads Stage 0: 1 · Pass 1: 4 · listed: all 4 Pass 1 DAGs · 4 priced · 1 cheaper than the selected plan (2) · each row: groups whose candidate differs from the selected plan’s choice. Row E3 reads G7 → #2 CountSketch+heap (pass1), costs 1.5, and is red. Clicking it sets the Stage 1 lane header to enumerated DAG E3 · G7 → #2 CountSketch+heap (pass1) · 11 nodes, draws that DAG, and sets the Stage 2 lane to no physical plan for an enumerated DAG.

Major algorithm changes

None.

Before this PR

The viewer showed one plan's three DAGs and the Stage 3 rejections. It could not show how many candidate DAGs the memo stands for, list them, or draw any of them.

After this PR

A document with memo.enumeration shows the counts per step and the listed DAGs, each described by how it differs from the selected choice, with cost and validity when priced. Any listed DAG can be drawn in the Stage 1 lane. Documents without memo look and behave as before.

Evidence

• Execution example: the usage above was run in headless Chromium (Playwright) against the page served locally. The header, the rows, the red E3 row (background rgb(251, 229, 227), --bad-soft), the lane 1 and lane 2 headers after the click, the row being released after a Stage 3 row click, and the hidden panel for plain Example 1 all matched the text above, with no page errors.
• Screenshot: not attached.

Verification

• Unit tests (node --test tools/dag-viewer/*.test.mjs, Node 22): enumeration.test.mjs, 12 tests:
  • documents without memo (or without enumeration) have no enumeration problems and no panel;
  • the priced fixture, and an unpriced copy of it, are valid;
  • each broken field (per-step entry, step, limit, choices, duplicate id, more choices than counted, choice map, DAG reference, booleans, cost, rejection, priced) is reported;
  • the header counts each step, with ≥ and the limit when counting stopped;
  • a row names the differing groups with origin and label, falls back to the bare index without memo.steps, and names every group when there is no selected choice;
  • sparse choices compare missing groups as candidate 0; a selected group the DAG does not read is named; a selected choice over a later step is flagged and marks no row;
  • priced rows carry cost, validity and the red marker; a choice with dag: null and error is accepted, shows the error and is disabled.
• App tests (python3 -m unittest test_viewer, stub DOM in V8): 81 tests, 8 skipped (they need a built stage_pipeline). New ones:
  • a document without memo keeps the panel hidden;
  • clicking an enumerated row redraws lane 1 with that DAG's nodes, clears lane 2, and marks the row; picking a Stage 1 candidate releases the row;
  • roots are named through roots_of in root-group order (swapped order is caught), a group answering q1 and q2 is both their root and stays bright when q2 is highlighted, and without the step's memo roots are root group #n;
  • on stage_pipeline: write the memo after each step, the selected choice and the enumeration #1003's examples/memo-ratio-of-quantiles.json, the panel says Pass 1: 49 and lists all 49 DAGs, and clicking a row redraws the Stage 1 lane with that DAG (root of q1) and clears Stage 2;
  • a broken enumeration refuses the document.
• Other checks: CI's Node step now also runs enumeration.test.mjs.

Architectural decisions

• The new code lives in its own file (enumeration.js), so it doesn't conflict with the memo view (DAG viewer: draw the memo after each step (groups with their candidates) #1005, memo.steps).
• The enumeration's check runs from app.js next to validateStagePipeline, not inside it, so stages.js stays unchanged.
• A choice's DAG has one root per root group, so the check does not require one root per query.

Limitations and follow-up

• selected_choice is over the last memo step. When the enumerated step is an earlier one, the comparison crosses steps (flagged in the header).
• A standalone scalar query has no root group, so an enumerated DAG of such a workload falls back to root group #n labels.

Human review — do not complete with an agent

• The MVP boundary is correct.
• New conceptual layers or public interfaces are necessary.
• The before/after description matches the intended product behavior.
• Human reviewer:
• Decision and rationale:

🤖 Generated with Claude Code