Repository navigation
Conversation
zzylol
force-pushed
the
refactor/1203-2a-5-scan-labels
branch
from
November 5, 2026 21:14
010cebc to
362d7d9
Compare
zzylol
force-pushed
the
refactor/1203-2a-row-k-terms
branch
from
November 5, 2026 21:19
e955c95 to
ef65e65
Compare
…ht per candidate (#1203) Stage 3's accuracy weight moves from the approximate candidate to each path of edges (u, c, i) through which a query reaches it: one producer read by a ratio's numerator and denominator has a term for each, each checked by the composition alone, pairs of terms checked together. Row 8 and the warm start constrain and count the paths a plan depends on (d[q, t]). The per-candidate map (accuracy_weights, accuracy_positions.rs) is deleted; AccuracyWeight is now per term. Interim until #1207 step 6 replaces the weights with row (K)'s constant lambda_{q,c,i}. Corpus and sketch-plan snapshots unchanged.
zzylol
force-pushed
the
refactor/1203-2a-5-scan-labels
branch
from
November 5, 2026 21:24
362d7d9 to
21aca01
Compare
zzylol
force-pushed
the
refactor/1203-2a-row-k-terms
branch
from
November 5, 2026 21:24
ef65e65 to
80a1092
Compare
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1222; part of #1203 (decision in the 2026-11-05 comment) and #1204 row (K), tracked in #1207.
Merge unit. This PR merges into main only together with the rest of #1207's merge unit (steps 1a–6, #1207 "Order"): interim items below are deleted by a later PR in the unit.
Why
#1203 decided (2026-11-05) that accuracy weights move from the candidate to how a query reaches it, as in row (K). Stage 3 gives each approximate candidate one weight per query, checked with the candidate's values approximate everywhere the query can read them. Once a state is one group read by several consumers (2a-6 makes a tumbling-window chain and its merge one group, shared by every consumer of the same state and width), a state shared by a ratio's numerator and denominator takes the stricter consumer's weight: it can no longer serve the numerator alone.
The same already happens on today's memo. In
q0.9 / q0.5 / q0.1(the three-quotient ofkept_builds_give_the_weights_of_fresh_builds), Pass 2's shared buildC1100(DDSketch, sized for 2 parts) is read by all three operands. Today it is unusable for the query, because the check makes it approximate in all three operands at once. A plan reading it for one or two operands, with the shares row bounding how many values it depends on, is valid.Interim encoding (decided 2026-11-05). Per-path terms are what 2a-6 needs: once a ratio's numerator and denominator read one shared merge group, a weight per candidate (or per (q, c, i) with values One / Unusable) cannot keep the numerator usable when the denominator is not. #1207 step 6, a later PR of the merge unit, collapses the terms to (q, c, i) with the constant propagation factor λ_{q,c,i} ($y_{u,c,i}$ .
ErrorPropagation, doc section 1), and deletesAccuracyTerm,AccuracyWeight,TermPath,accuracy_budget_parts, row 8,N[q]andA[q, parts]. This PR does not call a budget share λ: the per-term value isAccuracyWeight, and the dependence variables ared[q, t], not the formulation'sWhat
AccuracyTerm,AccuracyWeight). One producer reached through the numerator and through the denominator has two terms, with weightOneand weightUnusable.guarantee_check). Pairs are pairs of terms that one plan can depend on together.p[ctx, c]for each candidate on the path, crossing into a shareable candidate's context or anI_ras the realizer does, plus the slot linkeof each multi-slot input on it (SelectionMilp::term_path).depends_on). It skips a producer only when every term of the query with that producer is unusable.MilpInputs::accuracy_weights(a map per query keyed by candidate), candidate-keyedaccuracy_excluded_pairs,AccuracyWeight, andaccuracy_positions.rswith its per-operator-node grouping and its cycle-aware walk.UPDATE_SNAPSHOTS=1leaves every corpus and sketch-plan snapshot file unchanged. Planning time is 6.89 s → 6.24 s in total;ratio-of-quantilesdrops from 2.46 s to 1.60 s.New abstractions not in #1207 or the doc: decisions (2026-11-05)
Per #1207's interface policy, these names are not defined by #1207, #1203 or section 3 of
memo-groups-properties-requirements.md. #1207's steps 1a–6 merge into main as one merge unit (#1207, Order), so an interim abstraction may stay in this PR only when a later PR of the unit deletes it; none reaches main. Rows marked Pending approval are not deleted by any PR in the unit and need the user's decision.AccuracyTerm { output_reference, candidates, inputs, weight }+producer()AccuracyWeight { One, Unusable }(now per term)accuracy_budget_parts, row 8,N[q],A[q,parts])MilpInputs::accuracy_terms,accuracy_excluded_pairsover term indices,AccuracyTerms,accuracy_terms(),read_togetherTermPath+SelectionMilp::term_path; MILP variablesd[q, t](wasy[q, t])p[ctx, c]and slot linksep,eby #1204 PR 1)accuracy_positions.rs,AccuracyPositions, per-candidateaccuracy_weightsHow
accuracy_terms::paths_to_approximate_producers: a depth-first walk from each output candidate of each of the query's output references, down producers that have an approximate candidate below them, with no candidate twice on one path (links can close a cycle). Every approximate candidate it meets ends a term.Checker::check(&[&term])builds each output reference. A reference no term starts at reads exact stand-ins of its output candidates, as before. Otherwisebuilt(tails)builds the reference from the path tails below each candidate:with_guarantees);rebuilt): the tail's own node at the tail's input, and one exact stand-in at every other input.BuiltBefore). Terms through the same operator nodes are checked once.Pairs: terms with weight
Onewhose producers are sized for at least 2 parts, and that can be read together (read_together):Two paths through different candidates of one consumer input, or of one output reference, are never in one plan. Each such pair is built with both terms approximate; a failing pair is excluded.
accuracy_rows(row 8, A1 over the terms):I_ris forbidden, as before;not_all(indicators)for each unusable term, for every term when the target is exact, and for each excluded pair;d (q depends on a term)=ANDof a term's indicators, made only when the shares row is needed;(context, slot, input)along the path). Two terms with the same places are never both depended on, since each context holds one candidate per slot.Representation / rules / enumeration
Notation (formulation → code)
queryindex;MilpInputs::accuracy_terms[q]AccuracyTerm::producer(), the last ofAccuracyTerm::candidatesAccuracyTerm::candidates[d](consumercandidates[d + 1](its producer), andAccuracyTerm::inputs[d](inputAccuracyTerm(one per path of edges fromAccuracyTerm::weight: AccuracyWeight(One: one share of the budget;Unusable(reason))SelectionMilp::term_path→TermPath::indicators, whoseANDis the MILP variabled (q depends on a term)accuracy_rows(still #1144 row 8 withN[q],A[q, parts], until #1207 step 6)Concepts not in the formulation
AccuracyTerm::candidates/inputs). The formulation writesSummaryMerge, so their paths differ only above it.TermPath::places:(context, slot, input)along a path, the bound of the shares row. "Slot" and "context" are feat(plan-selection): group selection MILP over execution contexts as Stage 3's default search (#1095) #1144's, pending Stage 3 MILP: implement the formulation variables x, y, z, t and rows (A)–(I) as named code over explicit edges #1204.AccuracyTerm::output_reference): a query root's operator reference, as elsewhere in Stage 3.Key code interfaces added / modified
crates/plan-selection/src/groups.rs(crate-internal)crates/plan-selection/src/groups/accuracy_terms.rs(new; replacesaccuracy_positions.rs)crates/plan-selection/src/milp/model.rs(crate-internal)crates/plan-selection/src/milp/warm_start.rs:depends_on(milp, plan, query, term) -> bool(new);meets_accuracycounts the terms the plan depends on (was the runs it reaches).Usage.
q0.9 / q0.5 / q0.1(max_error 0.01), Stage 3's inputs printed from the test (Cn= physical candidate):Major algorithm changes
Before this PR
w(q, k)per approximate candidate: the check madekapproximate whereverqcould read it. Row 8 counted runs (context, candidate). The warm start skipped candidates with an unusable weight.After this PR
An accuracy weight per row (K) term (path of edges): row 8 constrains and counts the paths a plan depends on. The warm start checks the same paths.
Deleted
groups/accuracy_positions.rs:accuracy_positions,AccuracyPositions,pairs_that_meet,Walk,BuiltInWalk, the per-operator-node grouping of checks, the cycle handling of builds (a path never repeats a candidate, and off-path inputs are exact stand-ins, so no build crosses a cycle), and its testoutputs_built_across_a_cycle_are_not_kept(what it tested is gone).groups.rs:AccuracyWeight(nowAccuracyWeight, per term),MilpInputs::accuracy_weights, candidate-keyedaccuracy_excluded_pairs.model.rsrow 8: the per-rundep(p,a[q, k],y (depends on a run in m)), the(context, slot)places.warm_start.rs: the per-candidate weight lookups inallowedandmeets_accuracy.ratio_weights_come_from_the_compositionreads the weight per term, with the same expectations: at 1 part {1, missed}, at 2 parts {1}, no excluded pair.one_pane_chain_serves_a_ratio_and_its_operandsasks that every term with that producer has weightOne.kept_builds_give_the_weights_of_fresh_buildscompares terms and pairs. On the three-quotient it asserts that pair checks run (a 2-part producer with two weightOneterms read together), no longer that a pair is excluded. Today's excluded pairs there all pairC1103made approximate in three operands at once, which the shares row now excludes.accuracy_rows_follow_the_composition_on_a_ratiouses producers whose every term is unusable, and pairs of 2-part terms.terms_read_together_where_their_paths_can_meet.Plan cost, Stage 2 candidates and planning time
UPDATE_SNAPSHOTS=1left every corpus and sketch-plan snapshot file unchanged. Stage 2 is untouched (counts identical).--run-until stage3wall time per snapshot input, #1222 vs this PR, release builds on one host:Renames (2026-11-05)
PropagationFactor→AccuracyWeight, fieldlambda→weight, MILP variablesy (q depends on a term)→d (q depends on a term); rebased on #1215–#1222's restack.cargo fmt --all -- --check;cargo clippy --workspace --all-targets -- -D warnings: clean;cargo test -p asap-plan-selection --lib: 142 passed.Viewer
Viewer tests not updated (#1203 decision 6).
Verification
cargo fmt --all -- --check;cargo clippy --workspace --all-targets -- -D warnings: clean (on this head).cargo test -p asap-plan-selection --lib: 142 passed (on this head).Memo::add_candidate(a change of Stage 1 code only, same candidates):cargo test -p asap-integration-tests --test integration -- memo_enumeration reuse_rules operator_sharing sharing_issue_regressions finer_aggregate_reuse planner_layering_example having_shared_aggregate stage1_logical_asap_optimization joint_search_equivalence sketch_rollup rollup_controlled production_rollup_workloads: 154 passed, 23 ignored (pre-existing).cargo test -p asap-devtools --test devtools -- asap_planner:: asap_planner_acceptance:: memo_pricing:: viewer_contract::: 58 passed, 1 ignored (pre-existing).UPDATE_SNAPSHOTS=1 cargo test --release -p asap-devtools --test corpus_snapshots -- --include-ignored: 9 passed, no file changed.ratio_weights_come_from_the_compositionandkept_builds_give_the_weights_of_fresh_builds, which fail on 2a-6 with per-candidate weights, pass.Left
N[q]andA[q, parts]over terms.