Skip to content

refactor: define every stage's interfaces from #1207 and delete the ones they replace (#1207 PR 0) - #1226

Open
zzylol wants to merge 11 commits into
mainfrom
refactor/1207-interfaces
Open

zzylol wants to merge 11 commits into
mainfrom
refactor/1207-interfaces

Conversation

@zzylol

@zzylol zzylol commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Merge unit. This is PR 0 of #1207's merge unit (PRs 0–9). The PRs merge into main together, after PR 9. Each one is stacked on the one before it for review. This PR compiles, but planning panics at the first todo!(). Tests that reach a todo!() fail until the PR that fills it in (see "Test status").

Why

The memo, the properties, the edges and the Stage 3 variables of the optimization formulation have no named home in today's code:

  • a candidate's size lives inside its private root_node chain, sized for a budget share (accuracy_budget_parts);
  • timing and lifetime are folded into one Materialization enum;
  • edges are implicit in ProducerCandidates sets;
  • Stage 3 has no $y$.

#1207's new order (rewritten 2026-11-06) is interfaces first. This PR defines every data structure and interface of the concepts doc memo-groups-properties-requirements.md (sections 3–5, #1224 at afee671/f3687a055), with todo!() bodies. It also deletes the old interfaces and the code they replace. Each later PR fills in one stage behind these interfaces, so no PR needs an adapter between an old interface and a new one.

What

  • Types (asap-types). It adds every type and signature of the doc's section 3, and the section 4–5 types that Stage 3 uses:

    • one Memo, with candidates in MemoGroup::candidates;
    • CandidateExpression, LogicalProperties, WindowLayout, LogicalRequirements, ErrorPropagation, parameter_list, producer_satisfies_logical_requirements;
    • LogicalEdge, LogicalASAPMemoCandidates { memo, edges };
    • PhysicalProperties { timing, lifetime }, Lifetime, StorageMedium. There is no schedule property σ (decided 2026-11-06): a None or OneEvaluation result runs when the context that runs it runs, and every tumbling grid is anchored at phase 0;
    • PhysicalCandidate { logical_candidate_id, physical_properties }, PhysicalCandidate::delivers_on(memo, context), producer_satisfies_physical_requirements;
    • ExecutionContext, ExecutionContextId, RefreshWindowId (in asap_types::stages, so delivers_on can take the context);
    • PhysicalEdge, PhysicalCandidateKey, PhysicalASAPMemoCandidates::{edges, admissible_edges_into, admissible_edges_out_of};
    • SelectedEdge::input_index_of_consumer.

    Accessors are real code. Every function the doc assigns to PRs 1–8 is todo!().

  • Stage 0 is memo_of_queries (was generate_dag_of_logical_memo_groups). It returns a Memo whose candidate lists are empty; DagOfLogicalMemoGroups is deleted. The hash-consing code is unchanged.

  • θ is out of the operator (decided 2026-11-06). FieldDataType::Sketch is now Sketch(SketchAlgorithm, GroupingStrategy) (was Sketch(SketchKind, GroupingStrategy)).

    • ASAPOp::SummaryAgg's and SummaryTumblingWindowChain's family carries only the family, so k=200 and k=345 hash-cons to one group. θ lives only in LogicalProperties::parameters.
    • Code that read θ off a node reaches todo!() for the PR that brings θ from the candidate: the cost's state shape (PR 6), the executor's sketch factory and checks (PR 6), and the accuracy model's per-node sketch guarantee (PR 4).
    • Hydra's sizes are θ too (decided 2026-11-06). GroupingStrategy::SharedMultiSubpopulation is now { kind: HydraKind } (was { kind, params: HydraParams }), and the sizes are SketchParams::Hydra(HydraParams). Two Hydra sizes hash-cons to one group. HydraParams::per_subpopulation gives the per-subpopulation sketch's θ, which the error model and the state shape read.
    • SketchKind (family + θ) stays as a value type: the executor's sketch factory pairs the family with the node's parameters to configure weighted frequency.
  • Stage 1.

  • Stage 2. generate_physical_asap_memo_candidates is todo!() (PR 5) and Stage2AssignmentRealizer::realize is todo!() (PR 6). Everything else in the crate (catalog, timing split, reference pricer) is deleted.

  • Stage 3. milp/model.rs holds:

    • the baseline SelectionVariables { x, y, z, t } and the constraint functions (A)–(F);
    • the section 5 ExecutionContextVariables ($x^{\mathrm{mat}}$, $x_\kappa$, $y_\kappa$, $h$) and retention_constraints (R);
    • the section 4 AccuracyVariables ($E$, $F$, $D$, $H$) and accuracy_constraints (K).

    select_plan validates the cost inputs, then reaches todo!() (PR 6). The feat(plan-selection): group selection MILP over execution contexts as Stage 3's default search (#1095) #1144 MILP is deleted, with its groups, cost terms, warm start, completion, decode, LP diagnosis, checks and accuracy binding.

    • Assumptions (decided 2026-11-06), on h / need / (R). An edge's retention need is the time horizon the queries it serves read: a 5m range over 10 s windows needs 30 windows. It is a constant per edge, so (R) is linear. Query latency is ignored until the latency design: a result counts as read when its evaluation starts. PR 7, which implements (R), repeats this.
  • Realizer and export (decided 2026-11-06). The only way to export a plan is AssignmentRealizer::realize(&self, physical: &PhysicalASAPMemoCandidates, selected: &DagOfSelectedCandidates) -> Result<RealizedPlan, PhysicalCandidateError>.

    • RealizedPlan::plan.dag is the PhysicalASAPDAG, built directly, with each node's properties read from its PhysicalCandidate.
    • I used the existing trait method rather than a new free function because it already has this signature. Stage 2's Stage2AssignmentRealizer::realize is todo!() (PR 6).
    • Export from bare roots is deleted: compile_physical_asap_dag, compile_physical_asap_workload(_with_node_ids), PhysicalASAPDAGCompilation, PhysicalASAPNodeIdentityMap, PhysicalASAPDAG::with_every_node_at_query_time, ReferencePlanError, PlanOutput::physical_asap_dag, and asap_planner::test_support::query_time_reference (with the test-support feature).
    • installable_plan_of now takes the &RealizedPlan. Its tumbling-window expansion and plan_installable are todo!() (PR 6).
    • No side map holds per-node physical properties, and ExecutionDataStateError (the timing pass's error) is deleted.
    • The wire node carries lifetime in place of materialization, and θ as PhysicalASAPDAGNode::parameters: Option<SketchParams> (decided 2026-11-06): the build's candidate's LogicalProperties::parameters, None on every other node. PR 0 defines the field; PR 6's realizer fills it. INSTALLABLE_PLAN_WIRE_VERSION is 13. The backend changes with PR 6.
    • The executor's sketch factory reads θ from it: create_planner_accumulator(family, parameters: Option<&SketchParams>, input, grouping), which refuses a sketch build with None. The θ checks and kernels not wired to it yet stay todo!() (PR 6).
    • expand_tumbling_window_chains_in_workload is deleted (decided 2026-11-06): PR 6's realizer expands the realized DAG's chains. Its helpers tumbling_window and shifted are now #[cfg(any(test, feature = "test-support"))], because only tests call them.
    • Tests build PhysicalASAPDAG directly (decided 2026-11-06). The test-only asap_types::test_support::physical_asap_dag_at_query_time(root) and physical_asap_dag_at_query_time_with_node_ids(roots) export hand-built operator roots with every node at query time, not materialized and with parameters: None. Executor, runtime-install and frontend end-to-end tests use them instead of the deleted roots-based export.
  • Tools.

New abstractions not in #1207 or the doc (pending approval)

parameter_list and PhysicalCandidate::delivers_on are in the doc and approved, so they are not listed here.

Item Where Why it is needed Proposed status
LogicalPropertiesError::{ParametersOfAnotherFamily, ParametersOnExactOperator, WindowOnOperatorThatCannotBeWindowed, ZeroWindowWidth} types/src/stages/candidates.rs The doc names the error type without variants. These are one per rejection that section 3 lists for LogicalProperties::new. pending
LogicalPropertiesError::{SketchBuildWithoutParameters, ParametersNotInList} (ParametersNotInList was FloorNotInParameterList) same A sketch build needs θ, and a candidate's θ must be in parameter_list. approved 2026-11-06
LogicalRequirements::new(parameter_floor_for_required_accuracy) -> Self (was -> Result<Self, LogicalPropertiesError>) same A floor may be any value of the family's parameter type (e.g. heap_size ≥ k), so new has nothing to reject. approved 2026-11-06 (floor outside the list); infallible signature follows from it
PhysicalPropertiesError::OneEvaluationAtIngestion types/src/physical/properties.rs The doc names the error type without variants. This is the one row of section 2's "Invalid combinations" table. pending
QueryId(pub usize) (plan-selection/src/milp/model.rs), RefreshWindowId(pub usize) (types/src/stages/physical/execution_context.rs) as listed SelectionVariables::z ($z_{q,v}$) names QueryId and ExecutionContext names RefreshWindowId, but the doc does not define them. They index the workload's queries and its ingestion refresh windows. pending
ExecutionContext::Evaluation { recurrence: QueryRecurrence, range_evaluation: Option<RangeEvaluation> } (was QueryEvaluation(QueryId)); EvaluationSchedule and evaluation_schedule() deleted types/src/stages/physical/execution_context.rs (moved from plan-selection/src/milp/model.rs) Contexts reuse the workload's own types: queries with equal recurrence and range evaluation are evaluated together and share its OneEvaluation results; one-off queries are supported (ρ_κ = invocations). QueryRecurrence holds an f64, so contexts are deduplicated by equality and ExecutionContextId indexes the distinct contexts; no Hash. Moved to asap_types because PhysicalCandidate::delivers_on takes it. approved 2026-11-06
PhysicalASAPDAGValidationError::PhysicalProperties { node, reason } (was InvalidPhysicalProperties) types/src/ir/physical_export.rs Reads with its PhysicalPropertiesError reason. approved 2026-11-06
ExecutionContextVariables { x_materialized, x_in_execution_context, y_in_execution_context, retention_amount } plan-selection/src/milp/model.rs The doc gives the section 5 variables ($x^{\mathrm{mat}}$, $x_\kappa$, $y_\kappa$, $h$) as a table, not a struct. They are kept apart from the baseline SelectionVariables so that $y$ never has two meanings (section 5). pending
AccuracyVariables { error_bound, error_bound_of_input, failure_probability, failure_probability_of_input } same The doc gives $E_c, F_{c,i}, D_c, H_{c,i}$ (section 4) as formulas, not a struct. pending
retention_constraints, accuracy_constraints and the constraint function signatures (&PhysicalASAPMemoCandidates, &…Variables) -> Vec<good_lp::Constraint> same The doc says "signatures omitted". pending
#![allow(dead_code)] on milp/model.rs same select_plan does not build the MILP until PR 6, so nothing calls these yet. PR 6 removes the allowance. interim, deleted in PR 6–8
PhysicalASAPMemoCandidates::new(logical, candidates, query_roots, edges) and its private fields edges, admissible_edges_into_input, admissible_edges_out_of_candidate types/src/stages/physical/memo_candidates.rs The doc says "edges stored per (consumer, input); see edges()". The accessors return slices, so the indexes must be stored, and private fields need a constructor. pending
PhysicalCandidateKey { logical_candidate_id, physical_properties } same Kept per the 2026-11-06 decision. Listed because the doc shows it only in a comment. approved 2026-11-06
ErrorGeneratingPhysicalASAPMemoCandidates (one variant, Pruning(String)) physical-asap-optimization/src/catalog/mod.rs It existed before. Its other variants named deleted error types. existing, reduced
Pass2RuleError::{InvalidGroupReference, TooManyGroups, CyclicDependencies} logical-asap-optimization/src/pass2/memo/rules.rs The doc's rule signature returns Pass2RuleError. These are the existing variants that do not name deleted rules. existing, reduced
PhysicalASAPDAGNode::lifetime replaces materialization (wire version 11 → 13, with parameters below) types/src/ir/physical_export.rs Materialization is deleted, and the doc says the export writes timing and lifetime per node. pending (PR 6 changes the backend with it)
materialized_tumbling_window_chains (was kept_tumbling_window_chains) cost/src/lib.rs Renamed: "materialized", not "kept". rename
ScheduleGroup::queries (was readers), and the queries argument of schedule_groups / evaluation_rate cost/src/execution_demand.rs Naming sweep: a query, not a "reader". In the files this unit edits, "kept" for a materialized result or retained data is now "materialized" or "retained", and "reader" is "query" or "consumer". rename
PlanningReport loses operator_nodes and physical_producer_links planner/src/run.rs Both counted deleted concepts. Nothing is added. deleted
memo_of_queries (was generate_dag_of_logical_memo_groups) logical-asap-optimization/src/stage0.rs Renamed. approved 2026-11-06
FieldDataType::Sketch(SketchAlgorithm, GroupingStrategy) (was Sketch(SketchKind, …)) types/src/ir/schema/mod.rs θ is out of the operator. approved 2026-11-06
installable_plan_of(realized: &RealizedPlan, selection, entries) (was (roots, selection, tumbling_window_retention_ms, entries)) planner/src/installable.rs Export no longer takes bare roots. pending
#[allow(dead_code)] on the θ-based error model: estimators::sketch_guarantee, estimators::size_params, sizing::default_size_params, hll::classic_guarantee, univmon::{distinct_guarantee, incomplete_top_layer_statistic} accuracy/src/ Nothing reads θ off a node any more, so no production code calls them until PRs 1–4 compute ε per listed θ. Deleting them would delete the families' error model, which those PRs need. interim, removed in PR 1–4
pass1_memo, Pass1Error, PlanningRun::pass1_memo (were generate_per_computation_logical_memo_candidates, PerComputationLogicalMemoCandidatesError, PlanningRun::per_computation_candidates) Pass 1, planner/src/run.rs Renamed: they hold or return a Memo. approved 2026-11-06
expand_tumbling_window_chains_in_workload deleted types/src/ir/operator/tumbling_window_chain.rs No production caller after export from roots was deleted. approved 2026-11-06
PhysicalASAPDAGNode::parameters: Option<SketchParams> (wire version 12 → 13) types/src/ir/physical_export.rs θ is out of the operator, so the executor needs it per node. Filled in PR 6. approved 2026-11-06
SketchParams::Hydra(HydraParams); GroupingStrategy::SharedMultiSubpopulation { kind } (loses params) types/src/ir/schema/sketch.rs Hydra's sizes are θ. The direction is approved; the variant's shape (one Hydra variant wrapping HydraParams, rather than one variant per Hydra kind) is mine. direction approved 2026-11-06; shape pending
HydraParams::per_subpopulation(&self) -> SketchParams same The error model (sketch_guarantee) and the state shape (sketch_shape) read the emulated per-subpopulation sketch's θ, which GroupingStrategy used to carry. pending
create_planner_accumulator(family, parameters: Option<&SketchParams>, input, grouping) (gains parameters) executor/src/summary_kernels/factory.rs The factory reads θ from the exported node's parameters. pending
test-only builders asap_types::test_support::{physical_asap_dag_at_query_time, physical_asap_dag_at_query_time_with_node_ids} (feature test-support) types/src/test_support.rs, types/src/ir/physical_export.rs Keep the executor and runtime-install end-to-end tests by building PhysicalASAPDAG directly (decided 2026-11-06). Every node at query time, Lifetime::None, parameters: None. pending
#[cfg(any(test, feature = "test-support"))] on tumbling_window_chain::{tumbling_window, shifted} and on physical_export::grouping_compatibility types/src/ir/ Only tests and the test builder call them. pending
#[allow(dead_code)] on capability::valid_matrix and GroupStates::Keyed executor/src/ Their θ-based callers are todo!() until PR 6. interim, removed in PR 6

Ambiguities and questions

The 2026-11-06 answers are applied above: θ out of the operator, Pass 1 returns Memo, export via realize, memo_of_queries, x_materialized, Hydra sizes as θ, θ on the wire, the Pass 1 renames, deleting expand_tumbling_window_chains_in_workload, and tests that build PhysicalASAPDAG directly. The schedule σ is dropped (decided 2026-11-06): Schedule and its error variants are deleted, producer_satisfies_physical_requirements checks timing x lifetime only, and delivers_on is Stage 3's per-context alignment check. Also decided 2026-11-06: the renames agreed in review (#1224): LogicalPropertiesError, PhysicalPropertiesError, LogicalRequirements (parameter_floor_for_required_accuracy), producer_satisfies_logical_requirements, producer_satisfies_physical_requirements, StorageMedium, ExecutionContext/ExecutionContextId (x_in_execution_context, y_in_execution_context), and the fields logical_candidate_id, physical_properties, producer_candidate_id, consumer_candidate_id, logical_candidates, physical_candidates; a sketch build needs θ from its family's list; a requirement's floor may be outside the list; producer_satisfies_physical_requirements takes only the producer and consumer; an evaluation context is the workload's recurrence and range evaluation (ExecutionContext::Evaluation { recurrence, range_evaluation }), one-off queries included. No open questions; the items marked "pending" in the table above await approval.

Key code interfaces

All items below are new, except where marked:

  • MemoGroup gains candidates.
  • Memo loses candidates.
  • CandidateForMemoGroupId moved here from stages/physical/memo_candidates.rs.
  • OriginOfCandidateForMemoGroup loses SharedBuild and SummaryResult.
  • Memo::topological_order is unchanged in signature and reads the expressions' children.
  • ExecutionTiming is the existing enum, re-exported from physical::properties.
  • AssignmentRealizer and RealizedPlan are unchanged; realize is now the only export path.

Function bodies are shown as ;. The bodies of the functions the doc assigns to PRs 1–8 are todo!() in the code.

crates/types/src/stages/memo.rs

/// A group, by index in the memo's groups.
pub struct MemoGroupId(pub u32);

/// One distinct logical sub-expression of the workload.
pub struct MemoGroup {
    /// Its logical node as the frontends (and Pass 1 targets) see it; its
    /// children are other groups' nodes.
    pub logical_node: Rc<OperatorNode>,
    /// Its child groups, in the node's child order.
    pub child_groups: Vec<MemoGroupId>,
    /// The groups that read it ([`Self::parent_groups`]). Public only so
    /// Stage 1, which builds and extends the memo, can maintain it; later
    /// stages read it through [`Self::parent_groups`].
    pub parents: Vec<MemoGroupId>,
    /// The queries whose roots reach it ([`Self::reached_by_queries`]).
    /// Public only so Stage 1, which builds and extends the memo, can
    /// maintain it; later stages read it through
    /// [`Self::reached_by_queries`].
    pub queries: Vec<usize>,
    /// Its candidates: empty after Stage 0, filled by Stage 1.
    pub candidates: Vec<CandidateForMemoGroup>,
}

impl MemoGroup {
    /// The groups that read it.
    pub fn parent_groups(&self) -> &[MemoGroupId];

    /// The queries whose roots reach it, by index in the workload, ascending.
    pub fn reached_by_queries(&self) -> &[usize];
}

/// The groups some query's root reads, ascending: the groups of
/// `query_roots` that are a query's result or a scalar query's operand.
pub fn root_groups(query_roots: &[QueryRoot<MemoGroupId>]) -> BTreeSet<MemoGroupId>;

/// The queries whose root reads `group`, ascending.
pub fn queries_rooted_at(query_roots: &[QueryRoot<MemoGroupId>], group: MemoGroupId) -> Vec<usize>;

crates/types/src/stages/candidates.rs

/// One candidate: index `index` in group `group`'s candidates
/// ([`MemoGroup::candidates`]).
pub struct CandidateForMemoGroupId {
    pub group: MemoGroupId,
    pub index: usize,
}

/// The memo, Stage 0's and Stage 1's one type: the memo groups, each with
/// its candidates ([`MemoGroup::candidates`]: empty after Stage 0), and each
/// query's root over them.
pub struct Memo {
    pub groups: Vec<MemoGroup>,
    /// Each query's root over groups, by index in the workload.
    pub query_roots: Vec<QueryRoot<MemoGroupId>>,
}

impl Memo {
    /// Hash-consing: the group with this expression, added if new.
    pub fn find_or_add_group(&mut self, expression: Operator<MemoGroupId>) -> MemoGroupId;

    /// Adds `candidate` to `group`'s candidates.
    pub fn add_candidate(
        &mut self,
        group: MemoGroupId,
        candidate: CandidateForMemoGroup,
    ) -> CandidateForMemoGroupId;

    /// The candidate `id`.
    pub fn candidate(&self, id: CandidateForMemoGroupId) -> &CandidateForMemoGroup;

    /// The groups, each after every group it or one of its candidates
    /// reads, the lower index first among those ready: a dependency order for
    /// candidate traversal, whatever order rules appended groups in. `None`
    /// when those reads form a cycle, a group reading itself included.
    pub fn topological_order(&self) -> Option<Vec<MemoGroupId>>;
}

/// $v = (e_v, p^{\log}_v)$: one candidate of a memo group.
pub struct CandidateForMemoGroup {
    /// $e_v$.
    pub expression: CandidateExpression,
    /// $p^{\log}_v$.
    pub logical_properties: LogicalProperties,
    /// $r^{\log}_{c,i}$, one per input group: per child of
    /// `expression.operator`, in [`Operator::children`] order.
    pub input_requirements: Vec<LogicalRequirements>,
    /// The groups a fused computation also implements, beyond the group
    /// whose candidate it is; empty otherwise.
    pub fused_groups: Vec<MemoGroupId>,
    /// Which rule added it. Not in the formulation: kept for
    /// `stage_pipeline --text` and the viewer (approved 2026-11-05).
    pub origin: OriginOfCandidateForMemoGroup,
}

/// Which rule added a candidate. Nothing in planning reads it.
pub enum OriginOfCandidateForMemoGroup {
    /// Pass 1: exact, exact accumulator, or a summary family.
    Pass1,
    /// Canonical value preprocessing feeding a summary build.
    SummaryInput,
    /// A tumbling-window layout of a summary state group.
    TumblingWindows,
    /// A coarse group regrouping a finer group.
    Rollup,
    /// A top-k read from another group's ranking.
    TopKReuse,
    /// Its metric's scans widened to a shared label set, read through a
    /// `Project` (scan widening).
    WidenedScans,
}

/// $e_v = \operatorname{op}_v(G_{v,1}, \dots, G_{v,n})$: one operator, with
/// its arguments, whose children are groups. The input groups $G_{v,i}$ are
/// [`Operator::children`], in that order.
pub struct CandidateExpression {
    pub operator: Operator<MemoGroupId>,
}

/// $p^{\log}_v = (\boldsymbol\theta_v, \omega_v)$. The family is the
/// operator's argument, not a property. [`Self::new`] rejects parameters of
/// another family, not in the family's list, missing on a sketch build or
/// present on an exact operator, and a window on an operator that cannot be
/// windowed.
pub struct LogicalProperties {
    /// $\boldsymbol\theta_v$, from the family's predefined list; `None` for
    /// an exact operator.
    parameters: Option<SketchParams>,
    /// $\omega_v$.
    window_layout: Option<WindowLayout>,
}

impl LogicalProperties {
    pub fn new(
        expression: &CandidateExpression,
        parameters: Option<SketchParams>,
        window_layout: Option<WindowLayout>,
    ) -> Result<Self, LogicalPropertiesError>;

    /// $\boldsymbol\theta_v$.
    pub fn parameters(&self) -> Option<&SketchParams>;

    /// $\omega_v$.
    pub fn window_layout(&self) -> Option<&WindowLayout>;
}

/// $\omega_v$. The window count is the group's query range ÷ width;
/// retention is $h_v$ (Stage 3).
pub enum WindowLayout {
    Tumbling { width_ms: u64 },
}

/// Logical properties whose dimensions cannot go together: bad input to
/// [`LogicalProperties::new`].
pub enum LogicalPropertiesError {
    #[error("the parameters belong to another family than the operator's")]
    ParametersOfAnotherFamily,
    #[error("an exact operator has no summary parameters")]
    ParametersOnExactOperator,
    #[error("the operator cannot be windowed")]
    WindowOnOperatorThatCannotBeWindowed,
    #[error("a tumbling window has a positive width")]
    ZeroWindowWidth,
    #[error("a sketch build needs parameters from its family's list")]
    SketchBuildWithoutParameters,
    #[error("the parameters are not in their family's predefined list")]
    ParametersNotInList,
}

/// The family's predefined $\boldsymbol\theta$ list (#1207 step 6), each with
/// its error bound derived through `estimators::local_guarantee` for a given
/// statistic.
pub fn parameter_list(family: SketchAlgorithm) -> &'static [SketchParams];

/// $r^{\log}_{c,i}$: only what $G_{c,i}$'s identity does not fix.
pub struct LogicalRequirements {
    /// $\boldsymbol\theta^{\min}_{c,i}$: the weakest parameters whose
    /// $\varepsilon$, for the statistic the consumer reads, is at most
    /// $\text{cap}_{c,i}$ = min(validity cap of $g_c$, pruning cap), or a
    /// direct need such as heap_size ≥ k. Any value of the family's parameter
    /// type; it need not be in the family's list.
    parameter_floor_for_required_accuracy: Option<SketchParams>,
}

impl LogicalRequirements {
    pub fn new(parameter_floor_for_required_accuracy: Option<SketchParams>) -> Self;

    /// $\boldsymbol\theta^{\min}_{c,i}$.
    pub fn parameter_floor_for_required_accuracy(&self) -> Option<&SketchParams>;

    /// $g_c$'s linear bound: $\beta_{c,i}$ and $b_{c,i}$ per input, read
    /// from the operator and its arguments. An operator without an error
    /// contract is reported as unsupported; no factor is guessed.
    pub fn error_propagation(consumer: &Operator<MemoGroupId>) -> ErrorPropagation;
}

/// The local error propagation of one consumer: its output error is at
/// most $\sum_i \beta_{c,i}\varepsilon_{c,i}$ while each input's error stays
/// within its validity cap (concepts doc, section 4; approved 2026-11-05).
pub struct ErrorPropagation {
    /// $\beta_{c,i}$ per input, for that input's metric.
    pub factor_per_input: Vec<f64>,
    /// $b_{c,i}$ per input: the cap under which the linear bound holds;
    /// `None` when the bound holds everywhere (e.g. a ratio caps only its
    /// denominator).
    pub validity_cap_per_input: Vec<Option<f64>>,
}

/// $\alpha^{\log}_{u,c,i}$: $\boldsymbol\theta_u \succeq
/// \boldsymbol\theta^{\min}_{c,i}$, compared per family on the parameters
/// that set $\varepsilon$. No floor, or an exact producer, always satisfies.
pub fn producer_satisfies_logical_requirements(producer: &LogicalProperties, requirement: &LogicalRequirements) -> bool;

crates/types/src/stages/logical_memo_candidates.rs

/// $(u, c, i)$ over logical candidates: `producer_candidate_id` is a
/// candidate of $G_{c,i}$, the group input `input_index_of_consumer` of
/// `consumer_candidate_id` reads.
pub struct LogicalEdge {
    pub producer_candidate_id: CandidateForMemoGroupId,
    pub consumer_candidate_id: CandidateForMemoGroupId,
    pub input_index_of_consumer: usize,
    /// $\alpha^{\log}_{u,c,i}$.
    pub satisfies_logical_requirement: bool,
}

/// Stage 1, Pass 2's output.
pub struct LogicalASAPMemoCandidates {
    /// The memo after the last rule.
    pub memo: Memo,
    /// Every $(u, c, i)$ with $u \in G_{c,i}$, grouped by $(c, i)$.
    pub edges: Vec<LogicalEdge>,
}

crates/types/src/physical/properties.rs

/// Where a materialized output is stored.
pub enum StorageMedium {
    Memory,
    Disk,
}

/// $\ell_v$: how long a candidate's output lives. Only a materialized output
/// has a medium.
pub enum Lifetime {
    /// Discarded after its consumer's execution completes.
    #[default]
    None,
    /// Materialized until every consumer within one evaluation finishes.
    OneEvaluation(StorageMedium),
    /// Materialized across evaluations, so later evaluations reuse it.
    AcrossEvaluations(StorageMedium),
}

/// A timing and lifetime that cannot go together (the invalid combination
/// of the concepts doc, section 2).
pub enum PhysicalPropertiesError {
    #[error("an ingestion-time output cannot be materialized for one evaluation: there is no evaluation at ingestion time")]
    OneEvaluationAtIngestion,
}

/// $p^{\mathrm{phys}}_v = (\tau, \ell)$. [`Self::new`] rejects the invalid
/// combination (concepts doc, section 2).
pub struct PhysicalProperties {
    timing: ExecutionTiming,
    lifetime: Lifetime,
}

impl PhysicalProperties {
    /// The properties `(timing, lifetime)`, unless they cannot go together.
    pub fn new(
        timing: ExecutionTiming,
        lifetime: Lifetime,
    ) -> Result<Self, PhysicalPropertiesError>;

    /// $\tau_v$.
    pub fn timing(&self) -> ExecutionTiming;

    /// $\ell_v$.
    pub fn lifetime(&self) -> Lifetime;

    /// Whether $\ell_v \ne$ None. Derived, not stored.
    pub fn is_materialized(&self) -> bool;
}

crates/types/src/stages/physical/memo_candidates.rs

/// Index into [`PhysicalASAPMemoCandidates::edges`].
pub struct PhysicalEdgeId(pub usize);

/// One physical alternative of a logical candidate: what Stage 3 selects.
pub struct PhysicalCandidate {
    /// $e_v$ and $p^{\log}_v$.
    pub logical_candidate_id: CandidateForMemoGroupId,
    /// $p^{\mathrm{phys}}_v$.
    pub physical_properties: PhysicalProperties,
}

impl PhysicalCandidate {
    /// Whether a consumer running in `context` can read this producer (the
    /// concepts doc's alignment table, section 2). True for $\ell$ = None,
    /// and for `OneEvaluation` in the same context. A windowed
    /// `AcrossEvaluations` producer, with $w$ read from its logical window
    /// layout in `memo`, needs every evaluation time of the context on its
    /// grid: $w$ divides the context's evaluation grid (interval, range step
    /// and phase), each scheduled time, or a one-off query's time; false
    /// when the times are not known (an interval without a phase, an
    /// estimated rate, an unknown recurrence).
    pub fn delivers_on(&self, memo: &Memo, context: &ExecutionContext) -> bool;
}

/// $\alpha^{\mathrm{phys}}_{u,c,i}$: timing and lifetime only (the concepts
/// doc's table, section 2). Alignment is checked per context in Stage 3 with
/// [`PhysicalCandidate::delivers_on`].
pub fn producer_satisfies_physical_requirements(producer: &PhysicalCandidate, consumer: &PhysicalCandidate) -> bool;

/// $(u, c, i)$ over physical candidates.
pub struct PhysicalEdge {
    pub producer_candidate_id: PhysicalCandidateId,
    pub consumer_candidate_id: PhysicalCandidateId,
    pub input_index_of_consumer: usize,
    /// $\alpha^{\log}_{u,c,i}$.
    pub satisfies_logical_requirement: bool,
    /// $\alpha^{\mathrm{phys}}_{u,c,i}$.
    pub satisfies_physical_requirement: bool,
}

/// A physical candidate Stage 2 pruned, named by what it would have been:
/// candidates are renumbered after pruning, so a pruned one has no
/// [`PhysicalCandidateId`].
pub struct PhysicalCandidateKey {
    pub logical_candidate_id: CandidateForMemoGroupId,
    pub physical_properties: PhysicalProperties,
}

/// Stage 2's output, which Stage 3 selects from.
pub struct PhysicalASAPMemoCandidates {
    /// Pass 2's output: the memo, its candidates' $e_v$, $p^{\log}_v$,
    /// $r^{\log}_{c,i}$, and the logical edges.
    pub logical_candidates: LogicalASAPMemoCandidates,
    /// $\mathcal C$, indexed by [`PhysicalCandidateId`].
    pub physical_candidates: Vec<PhysicalCandidate>,
    /// Each query's root over groups, by index in the workload, passed
    /// through from Pass 2.
    pub query_roots: Vec<QueryRoot<MemoGroupId>>,
    /// The candidates Stage 2 pruned, in pruning order.
    pub pruned_candidates: Vec<PrunedCandidate<PhysicalCandidateKey>>,
    /// $\mathcal E$, grouped by (consumer, input); indexed by
    /// [`PhysicalEdgeId`].
    edges: Vec<PhysicalEdge>,
    /// $P_{c,i}$ after $\alpha$: the admissible edges into each (consumer, input).
    admissible_edges_into_input: HashMap<(PhysicalCandidateId, usize), Vec<PhysicalEdgeId>>,
    /// The admissible edges out of each candidate, by [`PhysicalCandidateId`].
    admissible_edges_out_of_candidate: Vec<Vec<PhysicalEdgeId>>,
}

impl PhysicalASAPMemoCandidates {
    /// Stage 2's output over `physical_candidates` and `edges` (grouped by
    /// (consumer, input)), with the admissible-edge indexes built once.
    pub fn new(
        logical_candidates: LogicalASAPMemoCandidates,
        physical_candidates: Vec<PhysicalCandidate>,
        query_roots: Vec<QueryRoot<MemoGroupId>>,
        edges: Vec<PhysicalEdge>,
    ) -> Self;

    /// $\mathcal E$: every edge, in (consumer, input) order.
    pub fn edges(&self) -> impl Iterator<Item = (PhysicalEdgeId, PhysicalEdge)> + '_;

    /// $P^{\mathrm{adm}}_{c,i}$: the admissible edges into input
    /// `input_index_of_consumer` of `consumer`
    /// ($\alpha^{\log} = \alpha^{\mathrm{phys}} = 1$).
    pub fn admissible_edges_into(
        &self,
        consumer: PhysicalCandidateId,
        input_index_of_consumer: usize,
    ) -> &[PhysicalEdgeId];

    /// The admissible edges out of `producer`: every consumer that may share it.
    pub fn admissible_edges_out_of(&self, producer: PhysicalCandidateId) -> &[PhysicalEdgeId];
}

/// The output references of `query_roots`: each operator reference, in
/// order (a scalar root's in `map_operator_refs` order), with the index of
/// the query whose root it is.
pub fn output_references<C: Clone>(query_roots: &[QueryRoot<C>]) -> Vec<(usize, C)>;

crates/types/src/stages/realizer.rs

/// Why a selected assignment could not be realized. Only rejected
/// assignments are safe to exclude from the finite search; adapter or
/// identity failures must stop with diagnostics, not become no-good cuts.
pub enum PhysicalCandidateError {
    #[error("selected physical assignment is infeasible: {0}")]
    RejectedAssignment(String),
    #[error("invalid or unsupported physical reconstruction: {0}")]
    InvalidOrUnsupported(String),
}

impl From<String> for PhysicalCandidateError {
    fn from(message: String) -> Self;
}
impl From<&str> for PhysicalCandidateError {
    fn from(message: &str) -> Self;
}

/// A realized selection (was `SelectedPhysicalCandidate`): the physical
/// plan Stage 2 built for it, how long each materialized tumbling-window chain of that plan
/// keeps its tumbling windows, and the operator node each selected node became.
pub struct RealizedPlan {
    /// The selection rebuilt as one physical plan, one operator node per
    /// selected node, each with its candidate's timing and lifetime.
    pub plan: PhysicalWorkloadPlan,
    /// For each tumbling-window chain of `plan` materialized beyond one evaluation
    /// (maintained at ingestion time, or built at query time and materialized
    /// across evaluations), by node id in `plan.dag`: how far back from the
    /// newest evaluation the runtime must keep its completed tumbling windows, in
    /// milliseconds. It is how far its tumbling windows reach back: the offset its
    /// newest tumbling window ends before the evaluation (`[60m] offset 10m`: 10 min),
    /// plus the chain's window, plus the longest range-evaluation span of
    /// its queries, so the earliest step of the next request still finds
    /// its tumbling windows; without an offset or range evaluations, just the window.
    /// Empty when the plan keeps no tumbling windows.
    pub tumbling_window_retention_ms: BTreeMap<PhysicalASAPNodeId, u64>,
    /// The operator node of `plan` each selected node became, indexed by
    /// [`SelectedNodeId`](super::SelectedNodeId).
    pub operator_node_of_selected_node: Vec<Rc<OperatorNode>>,
}

/// Realizes a [`DagOfSelectedCandidates`] through Stage 2, one plan node
/// per selected node. Implemented by the
/// Stage 2 crate, which holds the planning context (the workload's
/// original query roots, its entries and data, and whether query-time
/// results are retained); the facade builds it, passes it to Stage 3, and
/// calls the same method to build the selected plan.
///
/// Guarantees: a returned result has one operator node per selected node, each
/// rebuilt over its selected producers with its identity kept, guarantees
/// recomposed, and every output contract (result kind and schema) met; a
/// realized assignment also passes Stage 2's timing and dependency rules
/// and the physical DAG validator, with every node keeping its
/// candidate's timing and lifetime.
///
/// It checks structure and execution legality only. It does not check cost,
/// and it does not check accuracy targets: Stage 3 does both.
pub trait AssignmentRealizer {
    fn realize(
        &self,
        physical: &PhysicalASAPMemoCandidates,
        selected: &DagOfSelectedCandidates,
    ) -> Result<RealizedPlan, PhysicalCandidateError>;
}

crates/types/src/stages/physical/execution_context.rs (new)

/// A refresh window of the workload's ingestion pipelines, by index.
pub struct RefreshWindowId(pub usize);

/// Index of an [`ExecutionContext`] in the workload's list of distinct
/// contexts. Contexts are deduplicated by equality: [`QueryRecurrence`]
/// holds an `f64` (`EstimatedRate`), so a context cannot be hashed.
pub struct ExecutionContextId(pub usize);

/// $\kappa$: a class of executions with a common owner and schedule, where
/// a candidate with $\ell$ = None or `OneEvaluation` runs.
pub enum ExecutionContext {
    /// The evaluation of every query with this recurrence and range
    /// evaluation, the workload's own description of when it runs: queries
    /// with equal ones are evaluated together and share its `OneEvaluation`
    /// results. A and B both every 10 s at phase 0 give one context; a
    /// one-off query's context is its `OneTime` recurrence.
    Evaluation {
        recurrence: QueryRecurrence,
        range_evaluation: Option<RangeEvaluation>,
    },
    /// $I_r$: the ingestion pipeline of a refresh window.
    IngestionPipeline(RefreshWindowId),
    /// $\kappa_m$: inside an `AcrossEvaluations` candidate's own computation.
    Materialized(PhysicalCandidateId),
}

crates/plan-selection/src/milp/model.rs

/// A query, by index in the workload.
pub struct QueryId(pub usize);

/// The baseline variables (section 3).
struct SelectionVariables {
    /// $x_v$, by [`PhysicalCandidateId`].
    x: Vec<Variable>,
    /// $y_{u,c,i}$, on admissible edges only.
    y: HashMap<PhysicalEdgeId, Variable>,
    /// $z_{q,v}$, for $v \in G_q$.
    z: HashMap<(QueryId, PhysicalCandidateId), Variable>,
    /// $t_v$, by [`PhysicalCandidateId`].
    t: Vec<Variable>,
}

/// The execution-context variables (section 5), which replace $x$ and the
/// unindexed $y$ of the baseline.
struct ExecutionContextVariables {
    /// $x^{\mathrm{mat}}_v$, for an `AcrossEvaluations` candidate.
    x_materialized: HashMap<PhysicalCandidateId, Variable>,
    /// $x_{\kappa,v}$, for a candidate with $\ell$ = None or `OneEvaluation`.
    x_in_execution_context: HashMap<(ExecutionContextId, PhysicalCandidateId), Variable>,
    /// $y_{\kappa,u,c,i}$, indexed by the consumer's context.
    y_in_execution_context: HashMap<(ExecutionContextId, PhysicalEdgeId), Variable>,
    /// $h_v$, for an `AcrossEvaluations` producer.
    retention_amount: HashMap<PhysicalCandidateId, Variable>,
}

/// The accuracy variables of constraint (K) (section 4), all continuous.
struct AccuracyVariables {
    /// $E_c$, by [`PhysicalCandidateId`].
    error_bound: Vec<Variable>,
    /// $F_{c,i}$: the chosen producer's error bound on input $i$ of $c$.
    error_bound_of_input: HashMap<(PhysicalCandidateId, usize), Variable>,
    /// $D_c$, by [`PhysicalCandidateId`].
    failure_probability: Vec<Variable>,
    /// $H_{c,i}$: the chosen producer's failure probability on input $i$ of $c$.
    failure_probability_of_input: HashMap<(PhysicalCandidateId, usize), Variable>,
}

/// (A) $\sum_{u \in P_{c,i}} y_{u,c,i} = x_c$.
fn input_satisfaction_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (B) $y_{u,c,i} \le x_u$.
fn producer_selection_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (C) $\sum_{v \in G_q} z_{q,v} = 1$; (D) $z_{q,v} \le x_v$.
fn query_coverage_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (E) $t_u + 1 \le t_c + |\mathcal C|(1 - y_{u,c,i})$.
fn acyclicity_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (F) $x_v \le \sum_{(v,c,i)} y_{v,c,i} + \sum_q z_{q,v}$.
fn utilization_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &SelectionVariables,
) -> Vec<Constraint>;

/// (R) $h_v \ge \operatorname{need}_{\kappa,v,c,i}\, y_{\kappa,v,c,i}$.
///
/// Assumptions (decided 2026-11-06): an edge's need is the time horizon
/// the queries it serves read, a constant per edge (a 5m range over 10 s
/// windows needs 30 windows), so (R) is linear; query latency is ignored
/// until the latency design: a result counts as read when its evaluation
/// starts.
fn retention_constraints(
    physical: &PhysicalASAPMemoCandidates,
    variables: &ExecutionContextVariables,
) -> Vec<Constraint>;

/// (K): $E_c$ and $D_c$ per candidate, big-M on selected edges, and the
/// root checks $E_v \le \varepsilon_q + M_q(1 - z_{q,v})$,
/// $D_v \le \delta_q + (1 - z_{q,v})$.
fn accuracy_constraints(
    physical: &PhysicalASAPMemoCandidates,
    selection: &SelectionVariables,
    accuracy: &AccuracyVariables,
) -> Vec<Constraint>;

crates/logical-asap-optimization/src/pass2/memo/rules.rs

pub(crate) mod finer_aggregate;
pub(crate) mod rollup;
pub(crate) mod scan_labels;
pub(crate) mod summary_inputs;
pub(crate) mod topk;

/// Why a Pass 2 rule failed on a memo.
pub enum Pass2RuleError {
    #[error("memo contains an out-of-range group reference")]
    InvalidGroupReference,
    #[error("memo group capacity exceeds the group identifier range")]
    TooManyGroups,
    #[error("memo dependencies contain a cycle")]
    CyclicDependencies,
}

crates/types/src/ir/schema/mod.rs (excerpt)

/// What a schema field carries (the type stays).
pub enum FieldDataType {
    /// An ordinary, readable value.
    Plain(DataType), // unchanged
    /// Exact, mergeable accumulator state.
    ExactAggregate(ExactKind, ExactParams), // unchanged
    /// Approximate sketch state: its family and grouping layout. θ is a
    /// candidate's `LogicalProperties::parameters`.
    Sketch(SketchAlgorithm, GroupingStrategy), // was Sketch(SketchKind, GroupingStrategy)
    /// A summary sample (e.g. a reservoir).
    SummarySample(SamplingKind, SamplingParams), // unchanged
    /// Wavelet-transform summary state.
    Wavelet(WaveletKind, WaveletParams), // unchanged
    /// Fitted statistical-model summary state.
    StatModel(StatModelKind, StatModelParams), // unchanged
}

crates/types/src/ir/physical_export.rs (excerpt)

pub struct PhysicalASAPDAGNode {
    // id, payload, output_state, lifetime, output_schema, guarantee, retention_ms: unchanged
    /// θ of a summary build: its candidate's `LogicalProperties::parameters`.
    /// `None` on every other node and for an exact build. Filled in PR 6.
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub parameters: Option<SketchParams>, // new
}
pub const INSTALLABLE_PLAN_WIRE_VERSION: u32 = 13; // was 11 (in crates/planner/src/installable.rs)

crates/types/src/ir/schema/sketch.rs (excerpt)

pub enum SketchParams {
    // Kll, Cms, CountSketch, …: unchanged
    Hydra(HydraParams), // new: Hydra's sizes are θ
}
pub enum GroupingStrategy {
    PerSubpopulationInstance,
    SharedMultiSubpopulation { kind: HydraKind }, // was { kind, params: HydraParams }
}
impl HydraParams {
    pub fn per_subpopulation(&self) -> SketchParams; // new
}

crates/executor/src/summary_kernels/factory.rs and crates/types/src/test_support.rs (excerpts)

pub fn create_planner_accumulator(
    family: &SummaryFamilyType,
    parameters: Option<&SketchParams>, // new: the exported node's θ
    input: &SummaryUpdate,
    grouping: &GroupingStrategy,
) -> Result<Box<dyn AccumulatorUpdater>, String>;

// feature `test-support`: every node at query time, Lifetime::None, parameters None
pub fn physical_asap_dag_at_query_time(root: &Rc<OperatorNode>)
    -> Result<PhysicalASAPDAG, PhysicalASAPDAGValidationError>;
pub fn physical_asap_dag_at_query_time_with_node_ids(roots: &[QueryRoot])
    -> Result<(PhysicalASAPDAG, Vec<Rc<OperatorNode>>), PhysicalASAPDAGValidationError>;

Usage: not applicable yet. Every planning path reaches a todo!() (Pass 1 first). The first end-to-end usage example comes with PR 6.

Major algorithm changes

None added. Algorithms are deleted (see "Deleted"); later PRs re-implement them behind these interfaces.

Before this PR

The memo held private operator chains (root_node + input_groups) whose summaries carried θ in the operator, sized per budget share. Pass 2 copied consumers for shared builds and per-size groups. Stage 2 enumerated Materialization per operator node with interned producer sets. The #1144 MILP selected one candidate per result slot and context, with implicit edges. Export compiled any operator roots against a side map of physical properties.

After this PR

Every type and signature of the concepts doc's sections 3–5 exists with the doc's names:

  • θ lives only in LogicalProperties;
  • Pass 1 returns Memo;
  • the only export path is AssignmentRealizer::realize.

None of the old interfaces it replaces remains. Planning panics at the first unimplemented stage. PRs 1–8 fill in the stages in order, and PR 9 restores the tools and snapshots.

Interfaces per stage (doc section → code)

Doc section Interface Code
3, Stage 0 MemoGroupId, MemoGroup { logical_node, child_groups, parents, queries, candidates } crates/types/src/stages/memo.rs:16
3, Stage 0/1 Memo { groups, query_roots }, find_or_add_group, add_candidate, candidate crates/types/src/stages/candidates.rs:25
3, Pass 1 CandidateForMemoGroupId, CandidateForMemoGroup, CandidateExpression, LogicalProperties, WindowLayout, parameter_list, LogicalRequirements, producer_satisfies_logical_requirements crates/types/src/stages/candidates.rs
4 ErrorPropagation, LogicalRequirements::error_propagation crates/types/src/stages/candidates.rs
3, Pass 2 rule signature fn(&mut Memo) -> Result<(), Pass2RuleError> (rules 1–7), LogicalEdge, LogicalASAPMemoCandidates crates/logical-asap-optimization/src/pass2/, crates/types/src/stages/logical_memo_candidates.rs
3, Stage 2 ExecutionTiming (existing, re-exported), StorageMedium, Lifetime, PhysicalProperties crates/types/src/physical/properties.rs
3, Stage 2 PhysicalCandidateId, PhysicalEdgeId, PhysicalCandidate (+ delivers_on), producer_satisfies_physical_requirements, PhysicalEdge, PhysicalASAPMemoCandidates (+ edges, admissible_edges_into, admissible_edges_out_of), PhysicalCandidateKey crates/types/src/stages/physical/memo_candidates.rs
3, Stage 3 SelectionVariables { x, y, z, t }, constraints (A)–(F), SelectedEdge { producer, consumer, input_index_of_consumer }, DagOfSelectedCandidates crates/plan-selection/src/milp/model.rs, crates/types/src/stages/physical/graph.rs
5 ExecutionContextId, ExecutionContext, RefreshWindowId crates/types/src/stages/physical/execution_context.rs
5 $x^{\mathrm{mat}}, x_\kappa, y_\kappa, h$, (R) crates/plan-selection/src/milp/model.rs
4 $E, F, D, H$, (K) crates/plan-selection/src/milp/model.rs

Deleted, and what replaces each

Deleted Replaced by
Materialization (types/src/physical/materialization.rs) PhysicalProperties { timing, lifetime }
OperatorNode::{timing, materialization}, with_timing, with_materialization each PhysicalCandidate::properties (doc section 3, realizer)
PhysicalPropertiesAssignment/MaterializationAssignment side map, the timing pass timing.rs (apply_*, data_state, timing_under_consumer, validate_at_ingestion, planned_data_state, NodeToTimedNodeCache) realizer and export from PhysicalCandidate (PR 6)
PhysicalWorkloadPlan::physical_properties, TumblingWindowChainExpansion::physical_properties, PlanOutput::physical_properties same, PR 6
DagOfLogicalMemoGroups (+ group_for_node, roots_over_groups) one Memo
Memo::candidates (Vec<Vec<_>>) MemoGroup::candidates
CandidateForMemoGroup::{root_node, input_groups, accuracy_budget_parts} expression: CandidateExpression (children = input groups), logical_properties, input_requirements
OriginOfCandidateForMemoGroup::{SharedBuild, SummaryResult} hash-consing plus one state group per family; readouts are groups (PR 3)
Pass 1 memo_population.rs (try_populate_pass1, NodeToStructuralIdCache), size_for_failure_probability, budget-share sizing (accuracy_budget_parts_per_measure, compose_*_with_accuracy_budget_parts, Sizing::shares), PerComputationCandidates::walked Pass 1 candidates via find_or_add_group/add_candidate with parameter_list (PR 3)
pass2/asap_aware_cse.rs (node-level window, rollup, top-k and scan-widening rewrites) and pass2/finer_aggregate.rs Stage 0 hash-consing and find_or_add_group, with rules 1–7 over groups (PR 4)
rule 8 summary_results.rs Pass 1 makes the readout a group (PR 3); the doc deletes the rule
rule 9 shared_configuration_readers.rs, shared_producer.rs, shared_configurations.rs the state group holds every θ; $y$ chooses per consumer input
step 10 state_groups.rs (add_shared_summary_state_groups, per-size groups, check_unexpanded_memo) and reuse.rs one state group per family (PR 3/4)
pass2/memo/enumeration.rs (ChosenCandidatePerMemoGroup, enumerate, build_logical_asap_dags_from_chosen_candidates), asap_planner::selected_chosen_candidates doc Appendix A enumerator (PR 6 check)
OperatorNodeOfCandidateForMemoGroup(Id), PhysicalCandidate::{operator_node, timing, materialization, producer_candidates_per_input, may_produce_query_output}, PhysicalASAPMemoCandidates::{operator_nodes_of_candidates_for_memo_groups, implementation_roots, query_output_candidates, output_reference_count, physical_node, memo_group_of} PhysicalCandidate { logical_candidate_id, physical_properties }, edges(), admissible_edges_*
ProducerCandidates, ProducerSetInterner, producer_storage_metrics PhysicalEdge list (PR 5)
ReaderConstraint (all four variants) and reader_constraints the per-context alignment check delivers_on; the cost model (doc section 5, "No consumer-count policies")
may_keep_across_evaluations_per_sole_reader_query Lifetime in PhysicalProperties, chosen by Stage 3
Stage 2 catalog (configurations.rs incl. compatible_timing, factoring.rs, inventory.rs, pruning.rs, reconstruction.rs), split_by_timing.rs, materialization and ingestion-eligibility rules in lib.rs generate_physical_asap_memo_candidates and producer_satisfies_physical_requirements (PR 1, PR 5); realizer (PR 6)
reference pricer optimize_physical_asap(_chosen), features reference-pricer/test-support of the Stage 2 crate doc Appendix A enumerator (PR 6 check)
plan-selection reference_pricer, selector.rs, WholePlanSelection, feature reference-pricer same
groups.rs (ResultSlot, slot_of, result_slots, run_kind, execution contexts, dominance pruning), groups/accuracy_positions.rs ExecutionContext and $x_\kappa$/$y_\kappa$ (PR 7)
milp/model.rs (#1144: p[ctx,c], e binaries, schedule groups z[m,G], N[q], A[q, parts], constraint 8, constraint 9) SelectionVariables and (A)–(F) (PR 6), contexts (PR 7), (K) (PR 8)
milp/cost_terms.rs, warm_start.rs, completion.rs, decode.rs PR 6 (baseline objective, decode $y$), PR 7 (context costs, $h$)
milp/lp_diagnosis.rs, feature lp-diagnosis deleted, no replacement
checks.rs (post-selection accuracy and latency checks), accuracy_binding.rs (K) in PR 8; the latency check: deleted, no replacement (latency is out of scope)
SelectionModels deleted with its only users
devtools memo_pricing (reference_select, reference_enumeration, DEFAULT_REFERENCE_CHOICES, PlanningStagesRun::reference, the max_combinations/reference_choices parameters), fuzz SearchKind::Exhaustive and EXHAUSTIVE_MAX_COMBINATIONS doc Appendix A (PR 6)
AnalyticalCostError::PhysicalPropertiesError, PhysicalASAPDAGNode::physical_properties (both from the #1214 port) the wire validator checks (IngestionTime, OneEvaluation) directly
PerComputationLogicalMemoCandidates, Pass 1's PrunedWayToCompute / pruned_ways_to_compute / pruned_of_target, and PruneReason's Pass 1 variants (RequiresExact, NullableAvgInput, NoRelationalCountSketchRealization, UnsupportedByDeployment, QuantileOutsideUnitInterval, TopKHeapOverOneRowPerItem, NoErrorBoundForQueryTarget) Pass 1 returns Memo; showing what Pass 1 leaves out moves to #1205
SketchKind inside FieldDataType::Sketch; accuracy::sizing::sized (budget-share sizing); propagation::with_resized_summaries, sizing::{with_summaries_sized_for_failure_probability, with_summaries_sized_for_targets, per_input_failure_probability, sized_for_failure_probability} θ in LogicalProperties::parameters from parameter_list (PR 1/3)
roots-based export: compile_physical_asap_dag, compile_physical_asap_workload(_with_node_ids), PhysicalASAPDAGCompilation, PhysicalASAPNodeIdentityMap, PhysicalASAPDAG::{with_every_node_at_query_time, query_roots}, ReferencePlanError, PlanOutput::physical_asap_dag, ExecutionDataStateError, InstallablePlanError::Timing AssignmentRealizer::realize (PR 6)
asap_planner::test_support::query_time_reference and the planner's test-support feature deleted, no replacement yet: a query-time reference needs a realization at query time; PR 6 decides

Deleted tests

643 #[test] functions are deleted and 2 are added (PhysicalProperties::new). The per-file table, with what each file covered and which PR restores it, is in this comment. By group:

Group Deleted Restored by
#509 acceptance examples (planner_layering_example{1,2,3,4}) over the reference pricer 105 PR 6 (baseline, selection), PR 7 (materialization), PR 8 (accuracy)
plan-selection: whole-plan selector, #1144 MILP, result slots, checks 141 PR 6 (A)–(F) and decode, PR 7 contexts and (R), PR 8 (K); latency checks: no replacement planned
Pass 2 node-level rewrites, rule 9, step 10 per-size groups 61 PR 4 (rules 1–7); rule 9 and step 10: no replacement planned
Pass 1 private-node population, budget-share sizing, pruned reasons 34 PR 3; pruned reasons: #1205; budget split: no replacement planned
Stage 2 catalog, timing split, realization; the timing pass 52 PR 5, PR 6
Other planner-dependent tests: DAGs or plans from the planner, reuse rules, materialized views, reference pricing (executor, integration, planner, cost, types, frontend, devtools) 238 + 1 PR 6; materialized variants PR 7; tools PR 9
θ per node (state bytes, kernels, KLL equality across δ) 11 PR 1, PR 6

Kept by building the DAG directly (decided 2026-11-06). Tests that hand-built operator roots and ran them now export them with the test-only physical_asap_dag_at_query_time(_with_node_ids) instead of the deleted roots-based export. Restored: executor time_series_window, node_ids, current_series_heap, planspace_series_identity_heap, 3 of deployment_computation's tests; integration filtered_aggregates, kll_tumbling_window_execution, per_series_exact_execution, sql_cardinality, sql_frequency, exact_composition, and parts of sketch_rollup, stage1_logical_asap_optimization, operator_design_examples, operator_sharing, promql_numeric_regressions, sql_to_post_asap, promql_to_post_asap; types physical_export; frontend count_planning, pearson_corr. types/tests/physical_export.rs::untimed_plan_is_rejected (the +1 above, deleted after the comment was posted) is deleted: the builder always assigns query time, and an untimed export is no longer possible. Restored sketch tests fail at the executor's todo!() until PR 6, because the builder leaves parameters: None. Tests whose DAG came from the planner (runtime_install's planning tests, sql_per_group_topk, sql_table_scans, per_sample_updater, the planner-built deployment_computation tests, and others) stay deleted; PR 6 restores them.

Test status

cargo test -p <crate> --all-features --no-fail-fast at f57ae08, the same tree as the head except untimed_plan_is_rejected, which failed there and is deleted since; the asap-types row leaves it out. Except for the stale fixture below, every failure panics at a todo!().

Crate Passed Failed Ignored Fails at Passes after
asap-types 200 2 0 PhysicalProperties::new (the 2 new tests) PR 1
asap-accuracy 77 0 0 — —
asap-cost 31 1 0 the executor's θ check (capability.rs) PR 6
asap-logical-asap-optimization 95 13 0 the per-node sketch guarantee (10) and the distinct-count guarantee (3) PR 4
asap-physical-asap-optimization 0 0 0 no tests left PR 5 adds them
asap-plan-selection 3 0 0 — PR 6–8 add the MILP tests
asap-planner 9 19 1 Pass 1 (7), the sketch and distinct-count guarantees (12) PR 3 → PR 6
asap-executor 217 56 1 θ checks (capability.rs: 35 θ, 13 Hydra grid) and the weighted-frequency kernel (7); includes restored tests whose builder leaves parameters: None PR 6
asap-frontend-promql 221 6 1 Pass 1 (2), the sketch guarantee (4) PR 3–4
asap-frontend-sql 175 0 0 — —
asap-frontend-metricsql 18 0 0 — —
asap-devtools 49 48 3 Pass 1 and the sketch guarantee, most inside the asap_planner binary through the subprocess tests PR 6; text and viewer output in PR 9
asap-integration-tests 102 76 4 Pass 1 (37), the sketch guarantee (28), the distinct-count guarantee (7), the executor's θ check (3), and the stale fixture (1) PR 3–6; the fixture in PR 9
  • Stale fixture. materialized_views_equal_the_stored_outputs_main_listed reads a checked-in plan file in the old wire format (Sketch with θ). PR 9 re-baselines it with the snapshots.

Verification

  • cargo check --workspace --all-targets: clean, no warnings.
  • cargo clippy --workspace --all-targets: clean. The only allowances are the commented #[allow(dead_code)] markers listed under "New abstractions".
  • cargo fmt --check: clean.
  • After deleting untimed_plan_is_rejected: cargo test -p asap-types --all-features --test types physical_export (11 passed).
  • Tests per crate: see "Test status".
  • No snapshot or plan comparison applies: planning panics at Pass 1 until PR 3.

@zzylol

zzylol commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Per-file test deletions in #1226 (origin/main → PR head). D = file deleted, M = modified; counts are #[test] functions.

File main now deleted What it covered Restored by
cost/src/state_bytes.rs M 8 4 4 held memory / refresh runs over hand-assigned materialized DAGs; state bytes per θ PR 6–7
cost/tests/held_memory.rs D 1 0 1 held memory / refresh runs over hand-assigned materialized DAGs; state bytes per θ PR 6–7
cost/tests/ingestion_refresh_runs.rs D 7 0 7 held memory / refresh runs over hand-assigned materialized DAGs; state bytes per θ PR 6–7
devtools/src/candidate_labels.rs M 4 3 1 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
devtools/src/planning_stages_text.rs M 1 0 1 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
devtools/tests/asap_planner.rs M 46 43 3 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
devtools/tests/memo_pricing.rs D 6 0 6 reference pricing (memo_pricing), stage text, labels PR 6 (Appendix A check), PR 9 (tools)
executor/src/physical_planner/mod.rs M 3 1 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/src/summary_kernels/factory.rs M 7 5 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/deployment_computation.rs M 16 3 13 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/plan_properties.rs M 4 2 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/precompute_candidates.rs D 2 0 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/state_column_updater.rs M 7 5 2 executor runs of planner-built DAGs; kernels per θ PR 6
executor/tests/weighted_topk_binding.rs M 2 1 1 executor runs of planner-built DAGs; kernels per θ PR 6
frontend-promql/tests/univmon_candidates.rs M 2 1 1 planner-built plans exported and run PR 6
frontend-sql/tests/pearson_corr.rs M 7 6 1 planner-built plans exported and run PR 6
integration-tests/tests/declared_group_counts.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/deployment_capabilities.rs D 2 0 2 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/exact_composition.rs M 5 1 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/finer_aggregate_reuse.rs M 10 2 8 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/frequency_execution.rs D 1 0 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/having_shared_aggregate.rs D 1 0 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/materialization_e2e.rs D 4 0 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/materialized_views.rs M 4 2 2 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/memo_enumeration.rs D 3 0 3 per-group enumeration counts PR 6 (Appendix A)
integration-tests/tests/operator_design_examples.rs M 6 5 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/planner_layering_example1.rs D 31 0 31 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/planner_layering_example2.rs D 35 0 35 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/planner_layering_example3.rs D 22 0 22 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/planner_layering_example4.rs D 17 0 17 #509 example acceptance suite over the reference pricer (WholePlanSelection) PR 6 (baseline), PR 7 (materialization), PR 8 (accuracy)
integration-tests/tests/precompute_raw_samples.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/production_rollup_workloads.rs M 4 3 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/promql_to_post_asap.rs M 13 9 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/range_evaluation.rs D 6 0 6 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/reuse_rules.rs D 7 0 7 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/rollup_controlled.rs M 3 1 2 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/runtime_handoff.rs D 8 0 8 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sharing_issue_regressions.rs M 12 6 6 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sketch_rollup_planning.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_group_bound.rs D 7 0 7 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_to_physical.rs D 1 0 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_trailing_window.rs D 3 0 3 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/sql_two_percentiles.rs D 4 0 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/stage1_logical_asap_optimization.rs M 14 13 1 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/stage2_physical_asap_optimization_sql.rs D 4 0 4 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
integration-tests/tests/tumbling_window_merge.rs D 14 0 14 reference pricer, per-group enumeration, planner-built DAGs run end to end, reuse rules PR 4 (rules), PR 6 (selection, execution)
logical-asap-optimization/src/pass1/logical_memo_candidates.rs M 3 0 3 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass1/memo_population.rs D 13 0 13 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass1/per_computation_candidates.rs M 56 42 14 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass1/per_computation_candidates_tests.rs M 35 31 4 candidate population with private nodes, budget-share sizing, Pass 1 pruned reasons, θ in nodes PR 3; pruned reasons: #1205; budget split: no replacement planned
logical-asap-optimization/src/pass2/asap_aware_cse.rs D 27 0 27 node-level Pass 2 rewrites (windows, rollups, top-k, scan widening) PR 4
logical-asap-optimization/src/pass2/memo/rules/rollup.rs M 1 0 1 node-level Pass 2 rewrites (windows, rollups, top-k, scan widening) PR 4
logical-asap-optimization/src/pass2/memo/rules/shared_configuration_readers.rs D 2 0 2 rule 9 shared configurations no replacement planned (deleted by design)
logical-asap-optimization/src/pass2/memo/shared_configurations.rs D 6 0 6 rule 9 configuration enumeration no replacement planned
logical-asap-optimization/src/pass2/memo/state_groups.rs D 23 0 23 step 10 per-size state groups no replacement planned (deleted by design)
logical-asap-optimization/src/pass2/memo/window_candidates.rs M 2 0 2 node-level Pass 2 rewrites (windows, rollups, top-k, scan widening) PR 4
physical-asap-optimization/src/catalog/realize.rs M 1 0 1 Stage 2 catalog, materialization options, timing split, realization PR 5 (expansion), PR 6 (realization)
physical-asap-optimization/src/lib.rs M 24 0 24 Stage 2 catalog, materialization options, timing split, realization PR 5 (expansion), PR 6 (realization)
physical-asap-optimization/src/split_by_timing.rs D 4 0 4 Stage 2 catalog, materialization options, timing split, realization PR 5 (expansion), PR 6 (realization)
plan-selection/src/groups/accuracy_positions.rs D 1 0 1 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/groups/tests.rs D 11 0 11 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/lib.rs M 104 0 104 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/milp/model/tests.rs D 20 0 20 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/select/tests.rs D 4 0 4 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
plan-selection/src/selector.rs D 1 0 1 #1144 MILP, result slots, dominance, whole-plan selector and its checks PR 6 (A)–(F)/decode, PR 7 contexts/(R), PR 8 (K); latency checks: no replacement planned
planner/src/installable.rs M 4 2 2 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/src/pass.rs M 7 6 1 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/deployment_accuracy_model.rs D 3 0 3 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/per_sample_updater.rs D 2 0 2 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/runtime_install.rs M 42 5 37 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/sql_per_group_topk.rs D 6 0 6 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/sql_table_scans.rs M 11 3 8 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
planner/tests/summary_sharing.rs M 13 4 9 installed plans vs query_time_reference, plan_installable, reference pricer, Pass 1 pruned reasons PR 6 (install), PR 8 (accuracy model)
types/src/ir/properties/timing.rs D 23 0 23 timing pass over the side map PR 6
types/src/ir/schema/sketch.rs M 6 5 1 KLL equality across δ (θ in the type) PR 1
types/src/stages/physical/producers.rs D 2 0 2 producer-set interning no replacement planned
types/tests/materialized_view.rs D 10 0 10 export and materialized views with non-default physical properties; node timing PR 6
types/tests/physical_export.rs M 19 12 7 export and materialized views with non-default physical properties; node timing PR 6
types/tests/schema_rebuilding.rs M 9 7 2 export and materialized views with non-default physical properties; node timing PR 6
types/tests/structure_contract.rs M 7 6 1 export and materialized views with non-default physical properties; node timing PR 6
types/tests/summary_coverage.rs M 33 32 1 export and materialized views with non-default physical properties; node timing PR 6
types/tests/summary_merge_structure.rs M 9 8 1 export and materialized views with non-default physical properties; node timing PR 6
types/tests/tumbling_window_chain.rs M 14 10 4 export and materialized views with non-default physical properties; node timing PR 6
total 642

Restored in the 2026-11-06 revision through the test-only asap_types::test_support::physical_asap_dag_at_query_time(_with_node_ids) (every node at query time, unmaterialized, no θ): executor time_series_window, node_ids, current_series_heap, planspace_series_identity_heap, deployment_computation (3 of 15); integration filtered_aggregates, kll_tumbling_window_execution, per_series_exact_execution, sql_cardinality, sql_frequency, exact_composition, and parts of sketch_rollup, stage1_logical_asap_optimization, operator_design_examples, operator_sharing, promql_numeric_regressions, sql_to_post_asap, promql_to_post_asap; types physical_export; frontend count_planning, pearson_corr. Tests whose DAG came from the planner stay deleted (PR 6 restores them). Restored sketch tests fail at the executor's todo!() (θ from the node's parameters) until PR 6.

…iming, lifetime }

Deletes Schedule and the σ-only InvalidPhysicalProperties variants; the only
invalid combination left is (IngestionTime, OneEvaluation). satisfies_physical
checks timing x lifetime only; delivers_on is the per-context alignment check
(w | I and w | φ). A OneEvaluation result runs in one evaluation context, so
x_{κ,v} covers it as for ℓ = None.
…physical drops memo

InvalidLogicalProperties gains SketchBuildWithoutParameters and renames
FloorNotInParameterList to ParametersNotInList, returned by
LogicalProperties::new. LogicalRequirement::new no longer checks list
membership (a floor is any value of the family's parameter type, e.g.
heap_size >= k), so it returns Self. satisfies_physical takes only the
producer and consumer.
Context::QueryEvaluation(QueryId) becomes Context::Evaluation(EvaluationSchedule):
queries with the same (interval, phase) are evaluated together and share
their OneEvaluation results. evaluation_schedule returns that schedule.
QueryId stays for z_{q,v}. A non-periodic query has no context yet.
InvalidLogicalProperties -> LogicalPropertiesError, InvalidPhysicalProperties
-> PhysicalPropertiesError, LogicalRequirement -> LogicalRequirements
(parameter_floor -> parameter_floor_for_required_accuracy), satisfies_logical
/ satisfies_physical -> producer_satisfies_logical_requirements /
producer_satisfies_physical_requirements, Medium -> StorageMedium,
Context / ContextId -> ExecutionContext / ExecutionContextId
(x_in_context / y_in_context -> x_in_execution_context /
y_in_execution_context). PhysicalCandidate and PhysicalCandidateKey fields
-> logical_candidate_id, physical_properties; edge fields ->
producer_candidate_id, consumer_candidate_id; PhysicalASAPMemoCandidates
fields -> logical_candidates, physical_candidates.
@zzylol zzylol self-assigned this Nov 6, 2026
Deletes EvaluationSchedule and ExecutionContext::evaluation_schedule. An
evaluation context is Evaluation { recurrence, range_evaluation }: queries
with equal ones share it and its OneEvaluation results, one-off queries
included. ExecutionContext, ExecutionContextId and RefreshWindowId move to
asap_types::stages so PhysicalCandidate::delivers_on can take the context.
Contexts are deduplicated by equality (QueryRecurrence holds an f64).
PhysicalASAPDAGValidationError::InvalidPhysicalProperties becomes
PhysicalProperties { node, reason }. In the files this unit edits, "kept"
describing a materialized result or retained data becomes "materialized"
or "retained" (cost, materialized views, realizer, installable plan,
deployment capabilities; locals kept/raw_kept_rows -> retained/
raw_retained_rows), and "reader" becomes "query" where it is a query
(execution demand, ScheduleGroup::queries, cost usage, labels, tests) or
"consumer" where it is a consuming candidate.
find_or_add_group returns the node rules' SchemaDerivationError and
add_candidate a CandidateError (the node rules, or an output that differs
from the group's). Bodies stay todo!() until PR 2.
…mo (#1207 PR 0)

Memo::groups is private, read through groups(); groups are added only
through find_or_add_group and queries through add_query (todo!() until
PR 2). memo_of_queries returns Result and its body is todo!() until PR 2,
which builds Stage 0 on find_or_add_group; Stage 0's node-level CSE (cse.rs),
its test_support export and the dead share_identical helper are deleted.
…nd (#1207 PR 0)

MemoGroup holds expression: Operator<MemoGroupId> (its input groups are its
children), the schema and result kind derived when it is added, parents,
queries and candidates; logical_node and child_groups are deleted.
Memo::logical_node_trees rebuilds node trees on demand for the code that
still reads them (Pass 1 target discovery, devtools labels); todo!() until
PR 2.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CSE optimizer Workload optimization and plan selection refactor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant