Repository navigation
refactor(plan-selection): delete the old Stage 3 engine; select_plan runs only the group selection MILP - #1187
Merged
Merged
Conversation
…lection MILP is select_plan's only search (#1095) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
added a commit
that referenced
this pull request
Nov 3, 2026
…y and cost exactly (#1185, part 1) (#1188) * fix(plan-selection): run the group selection MILP to proven optimality and cost exactly (#1185) The group selection MILP stopped at wall-clock deadlines and costed some candidates inexactly past hardcoded caps, so the selected plan depended on machine speed and could differ from the optimum of the model. Deadlines removed: - select_plan no longer takes a deadline; the facade stops passing one (its own round budgets and the Pass 2 deadline are unchanged). - The warm start has no time budget: it tries workload order, its reverse and MORE_ORDERS pseudo-random orders, every run. - No per-part time split and no HiGHS time_limit: every part is solved to proven optimality. Any HiGHS status other than Optimal or Infeasible is a solver error. - Deleted WARM_START_MOST_TIME, MOMENT_TO_READ_THE_START_S, LP_TIME_LIMIT_S (the LP diagnosis's LPs and one-node root solve end on their own), SelectionMilpStatus, SelectionMilpError::{DeadlineReached, NoSolution}, SearchStatus::DeadlineReached, ErrorSelectingPlan::NoPlanByDeadline and the solution's relative_gap (always 0 now). Caps removed (every combination is costed and encoded exactly): - MAX_CONTEXTS, MAX_PRODUCER_CHOICES and UsageContext::TooManyContexts. - MAX_WINDOW_KINDS: every union of a scan run's window kinds is costed and encoded with exact indicators; the widest-window fallback is gone. - MAX_COMPOSITIONS: every composition is built and checked, so no set of approximate candidates is marked NotChecked for its size. HiGHS threads are now SelectionMilpOptions::highs_threads, default 1 (was 4), so ties between plans of equal cost break the same way. Snapshot changes: only the search status. o11y_bench q05, q19, q25 and the sketch workloads shared-quantiles-1h-mvp and quantile-single-mvp used to stop at the time limit with the optimal plan; they are now proven optimal (proven_optimal true, search_status optimal). Plans and costs are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(plan-selection): guard window-union enumeration against shift overflow; update deadline docs (review) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(plan-selection): name why candidates could not be costed when the selection MILP is infeasible The dag-viewer test test_a_plan_without_a_rate_fact_names_the_missing_fact failed on main since the group MILP became the default: candidates whose usage asap-cost could not price (e.g. a source without an arrival rate) were left out silently and the error said only 'infeasible'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(plan-selection): import AnalyticalCostError from asap-cost in the selector (main build after #1164 and #1187) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#1144 made the group selection MILP (#1095) Stage 3's default search. The old engine (candidate graph, cost bounds, the evaluator-backed MILP refinement loop and the bounded objective traversal) stayed behind
ASAP_STAGE3_SEARCH=old. It is no longer the plan we run, it is about 21.8k lines, and keeping it doubles every Stage 3 change.What
crates/plan-selection/src/search/(the old engine and its tests) and theASAP_STAGE3_SEARCHswitch.select_planmoves intocrates/plan-selection/src/select.rsand always runs the group selection MILP. Its signature is unchanged.selector.rs,WholePlanSelection) is now compiled only for tests and thereference-pricerfeature (its remaining users:stage_pipeline --exhaustive/--price-enumerated,planner_fuzz, integration tests).planner-layering.md's "How the search uses the memo" paragraph and code locations now describe the group selection MILP; the crate's cost-model docs sayselect_plansupports cost model 1 only (Memo search solver: MILP over the memo for all cost models #888).Capability lost, and where it is tracked
select::select_plan(group MILP, #1095)UnsupportedCostModel, as on main since #1144UnsupportedCostModelas on main since #1144ASAP_STAGE3_SEARCH=oldescape hatchHow
Move the entry point and its doc comment from
search/select.rsintoselect.rs(cost-input validation first, then the existing MILP path), drop the env switch,git rmsearch/, and gate the whole-plan selector oncfg(any(test, feature = "reference-pricer"))so the default build has no dead code.Key code interfaces added / modified
crates/plan-selection/src/select.rs,crates/plan-selection/src/lib.rscrates/plan-selection/src/report.rsUsage (unchanged for callers): the facade's
run_planningcallsasap_plan_selection::select_plan(&physical, &entries, &data, &checks, &cost_inputs, &realizer, deadline)and gets back theSelection(selected DAG,PlanCost, accepted accuracy per query, pruned candidates,SearchReport) exactly as on main with the default switch.Major algorithm changes
None. Main already ran the group MILP by default; this removes the non-default engine.
Before this PR
select_planran the group MILP unlessASAP_STAGE3_SEARCH=oldwas set, which ran the old candidate-graph engine (and was the only way to search under cost models 2/3).After this PR
select_planalways runs the group MILP.ASAP_STAGE3_SEARCHis ignored. Cost models 2/3 returnUnsupportedCostModeluntil #888.Verification
cargo check --workspace --tests: clean, no warnings; alsocargo check -p asap-plan-selection --tests --features reference-pricer.cargo test -p asap-plan-selection --lib: 142 passed, 0 failed.cargo test -p asap-devtools --test devtools stage_pipeline: 5 passed, 0 failed, 46 ignored (already#[ignore]on main: wall-clock or slow tests).cargo fmt --all.Limitations and follow-ups
stage_pipeline'ssearchfield,planner_fuzz --search candidate-connections) still names the deleted engine; rename in a follow-up.ErrorSelectingPlan::EvaluationDeadlineExceededis now produced only by the reference pricer; revisit with Remove unnecessary deadlines and caps in the group selection MILP #1185's deadline removal.joint-planning-search.md,joint-planning-implementation-plan.md, evaluation notes) still describe the old engine as history; not rewritten here.🤖 Generated with Claude Code