Skip to content

refactor(types, stages): every valid timing × lifetime state explicit; PhysicalRequirement, satisfies_physical, no named states - #1216

Closed
zzylol wants to merge 3 commits into
refactor/1200-physical-properties-off-logical-irfrom
refactor/1200-all-valid-states
Closed

refactor(types, stages): every valid timing × lifetime state explicit; PhysicalRequirement, satisfies_physical, no named states#1216
zzylol wants to merge 3 commits into
refactor/1200-physical-properties-off-logical-irfrom
refactor/1200-all-valid-states

Conversation

@zzylol

@zzylol zzylol commented Nov 5, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Stacked on #1223 (which is stacked on #1214); part of #1200, tracked in #1207. Base branch: refactor/1200-physical-properties-off-logical-ir.

Merge unit. This PR merges into main only together with #1207's steps 1a–6 (#1207, Order): the #1200 stack, #1203 2a and 2b, #1201 and #1204 PR 1. Nothing of the unit merges alone.

Representation / rules / enumeration

Why

The user asked that every state timing × lifetime can express be handled, except the invalid ones: each consumer gives each valid state a defined behavior, with no wildcard that lumps states together. In #1214 several consumers matched (timing, lifetime) with wildcards ((_, Lifetime::None), (IngestionTime, _)), so a state could take another state's behavior silently.

What

  • Exhaustive matches on (timing(), lifetime()). Each of the 8 valid states has its own arm, or an arm that lists its states. The invalid pair has one arm, unreachable!("rejected by PhysicalProperties::new"). Consumers: PhysicalProperties::new, is_materialized and satisfies_physical; Stage 2's summary_properties_allowed; Stage 3's run_kind; asap_cost::node_usage (rows per window, retained bytes, working memory, maintained); materialized_tumbling_window_chains; materialized_run_rate; the devtools phase_label.

  • One shared classification. asap_cost::execution_demand::materialized_run_rate(PhysicalProperties) -> Option<MaterializedRunRate> (PerSecondOfArrivingData, PerEvaluation, PerRequest; None when not materialized) is the one place that says how often a materialized output is computed. computations stores it per materialized node and matches on it directly; the MILP's context_rates uses it too.

  • Interfaces that held only materialized values take the full PhysicalProperties (user decision, 2026-11-05; no restricted type, no parallel enum):

    • PhysicalPropertiesAssignment::set(node, properties) already stores any valid properties since refactor(types, stages): physical properties off the logical IR (OperatorNode has no timing or lifetime) #1223 (a not-materialized entry is an ordinary placement at its timing; the debug_assert went there).
    • MaterializedOutputCandidate::options() keeps Vec<Option<PhysicalProperties>>: each option is the assignment entry the node would get, and None is "no entry". That is not a second spelling of (τ, None): the not-materialized choice fixes no timing, since the timing pass places an unassigned node where its consumer runs (a shared TimeRange or Scan read by ingestion work runs at ingestion time). Replacing None by (QueryTime, None) pinned such nodes to query time and failed two runtime_install tests (RealizerRejected for a requested (IngestionTime, None)), so the type stays; its documentation now says what None means.
    • RunKind::Materialized and ContextKind::Materialized(m) carry no copy of the candidate's properties; context_rates looks them up in physical.candidates[m]. A κ_m context whose candidate is not materialized is a defined error (SelectionMilpError::Cost), not a fallback rate.

  • Names follow the formulation. RunKind::Shareable → RunKind::Materialized and ContextKind::Shared → ContextKind::Materialized (κ_m, Stage 3 MILP: implement the formulation variables x, y, z, t and rows (A)–(I) as named code over explicit edges #1204: contexts index only ℓ = None candidates; a materialized candidate has one x_v). The rest of Stage 3 follows: context_of_shareable → context_of_materialized, ingestion_context_of_shareable → ingestion_context_of_materialized, shared_readers → materialized_consumers, shareables_readers_first → materialized_consumers_first, shareables_needing_exact_reach → materialized_needing_exact_reach, Scope::SharedContexts → MaterializedContexts, Run::Shared → Run::Materialized, GroupRead::shareable → materialized. Labels: "query time, shared" → "query time, one evaluation", "query time, kept" → "query time, materialized across evaluations" (one sketch-plan snapshot's display_name changes with it).

  • Errors name their node. materialized_tumbling_window_chains returns Result<_, (PhysicalASAPNodeId, AnalyticalCostError)>, as plan_cost does, so plan selection reports it through the existing ErrorSelectingPlan::Cost { candidate, node, error }. Stage 3's window_read_through matches the chain's (timing, lifetime) itself; the cost crate's predicate stays private.

  • Fewer error names (user decision, 2026-11-05). PhysicalRequirement::new's (IngestionTime, Later) case is InvalidPhysicalProperties::LaterEvaluationAtIngestion (no separate requirement error type). Cost reports a node whose pair is invalid through the export's own variant, PhysicalASAPDAGValidationError::InvalidPhysicalProperties { node, reason }, wrapped as AnalyticalCostError::InvalidPhysicalDag(_), instead of a parallel AnalyticalCostError::InvalidPhysicalProperties. Error names after this PR: InvalidPhysicalProperties::{OneEvaluationAtIngestion, LaterEvaluationAtIngestion}; PhysicalASAPDAGValidationError::InvalidPhysicalProperties { node, reason }; AnalyticalCostError::InvalidPhysicalDag(PhysicalASAPDAGValidationError); export's edge errors as on refactor(types, stages): physical properties off the logical IR (OperatorNode has no timing or lifetime) #1223 (feat(types): an unmaterialized summary build runs at its consumer's timing; one α^phys error at export #1219 folds them into one α^phys failure).

  • Inlined (user decision, 2026-11-05): PhysicalASAPDAGNode::physical_properties() (callers build PhysicalProperties::new(node.output_state.timing, node.lifetime)), select::cost_error, PhysicalPropertiesAssignment::lists_every_node_of and the facade's debug assertion that used it, PhysicalProperties::not_materialized_at.

  • The realizer checks the whole selected state. After the timing pass, realize_selected_candidates compares each selected node's realized PhysicalProperties with the one Stage 3 selected and priced (before: the lifetime only), so a None option the timing pass places at the other timing is an error.

  • The warm start uses materialized_run_rate for a κ_m context's rate (per second at ingestion time, per evaluation, or per request), as computations and the MILP's context rates do.

  • Comment at Stage 2's generation site lists the valid states it does not offer, with the reasons below.

  • Table tests over every valid state. Each test enumerates the 10 timing × lifetime pairs and keeps those PhysicalProperties::new accepts (the 8 valid states), inline; there is no shared test helper. The integration-tests helpers default_physical_properties / physical_properties_at_ingestion (refactor(types, stages): physical properties off the logical IR (OperatorNode has no timing or lifetime) #1223) are inlined into their tests too.

    Consumer Test
    new properties::tests::exactly_the_ingestion_time_one_evaluation_pairs_are_invalid (all 10 pairs: exactly the 8 succeed, the 2 invalid fail)
    satisfies_physical properties::tests::satisfies_physical_over_every_valid_state (8 states × 3 valid requirements)
    is_materialized properties::tests::is_materialized_over_every_valid_state
    PhysicalRequirement::new properties::tests::later_evaluation_at_ingestion_is_the_invalid_requirement
    export validation crates/types/tests/physical_export.rs::export_validation_over_every_valid_state
    cost plan-selection lib cost_over_every_valid_state
    maintained partial_copies_over_every_valid_state
    execution demand computations_over_every_valid_state
    run rate asap_cost materialized_run_rate_over_every_valid_state
    run kind groups::tests::run_kind_over_every_valid_state
    devtools labels candidate_labels::tests::phase_label_over_every_valid_state
    executor physical planner crates/executor/tests/every_valid_state.rs

    Each table test lists its expected value per state, not a copy of the implementation's match. Limits (review): cost_over_every_valid_state checks that each state prices finitely and that exactly the Disk states add disk bytes, not each state's bytes; the executor table puts a requested (IngestionTime, None) build where the timing pass places it (query time) until feat(types): an unmaterialized summary build runs at its consumer's timing; one α^phys error at export #1219; summary_properties_allowed and context_rates have no table test of their own.

Deleted:

State × consumer → behavior

ing = IngestionTime, qry = QueryTime; M = Memory, D = Disk.

(τ, ℓ) satisfies_physical, same evaluation (consumer qry / ing) Stage 2 offers Export validation run_kind Cost (node_usage, materialized_run_rate) Executor physical planner
(ing, None) false / true non-summary nodes (summary builds: #1219) valid; for a summary build or MaintainPopulation, rejected here; accepted in #1219 Ingestion working memory at peak; computed per refresh window computed in the ingestion run that reads it
(ing, AcrossEvaluations(M)) true / true summary builds when ingestion is eligible valid Materialized, per second of arriving data chain: retained windows + 1; other: state × 2; per-worker newest window materialized view slot, refreshed at ingestion
(ing, AcrossEvaluations(D)) true / true no: #1217 valid Materialized, per second same bytes on disk; per-worker newest window in memory view slot refreshed at ingestion; executes in memory (#1217)
(qry, None) true / false every node valid PerContext working memory held per consumer's evaluation computed in the query run
(qry, OneEvaluation(M)) true / false summary builds; non-summary nodes two queries reach valid Materialized, per evaluation state held until the slowest reader finishes computed once in the query run
(qry, OneEvaluation(D)) true / false no: #1217 valid Materialized, per evaluation the state on disk computed once in the query run; kept in memory (#1217)
(qry, AcrossEvaluations(M)) true / false tumbling-window chains only, with query-time retention valid Materialized, per request chain: retained windows − 1 between evaluations; other: state × 2 view slot refreshed on first read
(qry, AcrossEvaluations(D)) true / false no: #1217 valid Materialized, per request same bytes on disk view slot refreshed on first read; executes in memory (#1217)

Valid states Stage 2 does not offer, and why (also in the comment at the generation site):

  1. Any Disk medium (Executor: store Disk-medium materialized views and one-evaluation results on disk #1217). The executor has no disk store; a Disk state given to it executes in memory, while cost prices it as disk. Decided 2026-11-05: no Disk offer until Executor: store Disk-medium materialized views and one-evaluation results on disk #1217.
  2. (IngestionTime, None) for a summary build. The timing pass runs an unmaterialized build at query time, and export rejects the pair; feat(types): an unmaterialized summary build runs at its consumer's timing; one α^phys error at export #1219 changes the placement, and the PR stacked on feat(types): an unmaterialized summary build runs at its consumer's timing; one α^phys error at export #1219 offers it.
  3. (QueryTime, AcrossEvaluations) for anything but a tumbling-window chain. Only a chain reuses what earlier evaluations built.

(IngestionTime, AcrossEvaluations) is likewise not offered for non-summary nodes: rows are not maintained at ingestion time.

Formulation notation → code

Formulation Code (introduced or renamed here)
rate of a materialized $v$ (from $\sigma_v$ in #1200 PR 2; today from timing and lifetime) MaterializedRunRate, materialized_run_rate
run kind, $\ell \ne$ None RunKind::Materialized (was Shareable)
$\kappa_q$, $I_r$, $\kappa_m$ ContextKind::{Query, Ingestion, Materialized} (Materialized was Shared)
$r^{\mathrm{phys}}_{c,i}$ PhysicalRequirement { consumer_timing, reads_from: Evaluation, consumer_schedule: Option<EvaluationSchedule> } (new)
$(I_\kappa, \varphi_\kappa)$ of the consumer's context, from its query's $\mathcal T_q$ EvaluationSchedule { interval_ms, phase_ms } (new; None everywhere until #1200 PR 2)
invalid $(\tau, \ell)$ or (IngestionTime, Later) InvalidPhysicalProperties::{OneEvaluationAtIngestion, LaterEvaluationAtIngestion}
$\alpha^{\mathrm{phys}}_{u,c,i}$ (timing and lifetime table) satisfies_physical(&producer, &requirement) (was readable_at)
materialized($v$) $\iff \ell_v \ne$ None PhysicalProperties::is_materialized (was Lifetime::is_materialized)

Concepts not in the formulation (for the user to name or add to #1200 / #1204):

New abstractions not in #1207 or the doc: decisions (2026-11-05)

Per #1207's interface policy; the user decided each row on 2026-11-05. #1207's steps 1a–6 are one merge unit, so an interim abstraction may stay only when a later PR of the unit deletes it.

Abstraction What it is for Decision
MaterializedRunRate { PerSecondOfArrivingData, PerEvaluation, PerRequest }, materialized_run_rate(p) how often a materialized candidate runs, from (τ, ℓ); shared by computations, the MILP's context rates and its warm start Approved until #1200 PR 2, which replaces it with the schedule's rate $\rho_{\sigma_v}$
RunKind::Materialized (was Shareable), ContextKind::Materialized (was Shared) and the κ_m renames in Stage 3 the formulation's materialized(v) and $\kappa_m$ Kept (#1200's text now says Materialized; ContextKind::Materialized matches the doc's Context::Materialized)
cost_error maps (node, AnalyticalCostError) to ErrorSelectingPlan::Cost Inlined in this PR
is_query_time_materialized_tumbling_window_chain chain whose scan reads one window Inlined in this PR (private in asap-cost; Stage 3 matches the pair itself)
PhysicalProperties named consts, not_materialized_at named valid states Deleted in this PR: PhysicalProperties::new(τ, ℓ)
PhysicalASAPDAGNode::physical_properties() the export node's pair Inlined in this PR
InvalidPhysicalRequirement, AnalyticalCostError::InvalidPhysicalProperties requirement error; cost's copy of the export's bad-pair error Merged in this PR: InvalidPhysicalProperties::LaterEvaluationAtIngestion; AnalyticalCostError::InvalidPhysicalDag(PhysicalASAPDAGValidationError) carries the export's variant
AnalyticalCostError::InvalidPhysicalDag(PhysicalASAPDAGValidationError) cost's one way to report an invalid plan node, carrying the export's error Pending approval (new name from the merge)
PhysicalProperties::every_valid, default_physical_properties, physical_properties_at_ingestion (test-only) table tests and fixtures Inlined in this PR
PhysicalPropertiesAssignment::materialized_at_ingestion_where_allowed (test-support) "materialize every eligible build at ingestion time", used by 8 test sites and validate_at_ingestion / planned_data_state Interim, deleted with the side map by #1201 + #1204 PR 1. Not inlined: it walks the timing pass's private rules (forces_query_time, is_filtered_build); it replaces main's MaterializationAssignment::with_default
PhysicalPropertiesAssignment and its fields on PhysicalWorkloadPlan, PlanOutput, WholePlanSelection, TumblingWindowChainExpansion; SelectedPlan; options() -> Vec<Option<_>>; Stage 2's whole-assignment renames per-node properties of the realized OperatorNode DAG Interim, deleted by #1201 + #1204 PR 1 (see #1214, #1223)

Names: consumer, not reader (user decision, 2026-11-05). The names this stack introduces say consumer (the candidate reading an input) or query (whose schedule $\mathcal T_q$ an execution context κ runs on): consumer_timing, consumer_schedule, EvaluationSchedule, materialized_consumers, materialized_consumers_first. Existing names that later PRs of the unit delete keep their word: ReaderConstraint (deleted by #1204 PR 1), may_keep_across_evaluations_per_sole_reader_query and MinimumQueryReaders (deleted by #1200 PR 2 / #1204 PR 1).

Replaced by the defined interface in this PR: PhysicalProperties::readable_at → PhysicalRequirement + satisfies_physical; Lifetime::is_materialized → PhysicalProperties::is_materialized. Evaluation::Later has no production consumer yet (every edge today reads the same evaluation); the table test covers it.

Divergences from the doc's section 3: satisfies_physical checks the timing-and-lifetime table only; the schedule part needs σ (#1200 PR 2).

How

Consumers match on (p.timing(), p.lifetime()). Where several consumers need the same classification (how often a materialized output runs), it lives in one function, materialized_run_rate. Readers of exported plan nodes build their pair with PhysicalProperties::new(node.output_state.timing, node.lifetime); an invalid node is reported, never skipped or relabelled.

Key code interfaces added / modified

crates/types/src/physical/properties.rs

impl PhysicalProperties {
    // Type unchanged from #1214: struct { timing: ExecutionTiming, lifetime: Lifetime }.
    // pub const NOT_MATERIALIZED_AT_QUERY_TIME, NOT_MATERIALIZED_AT_INGESTION,
    //   MATERIALIZED_IN_MEMORY_FOR_ONE_EVALUATION,
    //   MATERIALIZED_IN_MEMORY_ACROSS_EVALUATIONS_AT_INGESTION,
    //   MATERIALIZED_IN_MEMORY_ACROSS_EVALUATIONS_AT_QUERY_TIME: Self;  // removed: use new
    // pub fn not_materialized_at(timing) -> Self;                       // removed: use new
    /// materialized(v): lifetime OneEvaluation or AcrossEvaluations.
    pub fn is_materialized(self) -> bool;                       // new; was Lifetime::is_materialized
    // pub fn readable_at(self, reader: ExecutionTiming) -> bool; // removed: satisfies_physical
}
// impl Lifetime { is_materialized, is_across_evaluations }    // removed

/// Bad input to PhysicalProperties::new or PhysicalRequirement::new; stays.
pub enum InvalidPhysicalProperties {
    /// (IngestionTime, OneEvaluation): no evaluation at ingestion time.
    OneEvaluationAtIngestion,    // unchanged
    /// (IngestionTime, Later): an ingestion-time consumer has no later evaluation.
    LaterEvaluationAtIngestion,  // new
}

/// Which evaluation's output a consumer reads. New (doc section 3).
pub enum Evaluation {
    /// The consumer's own evaluation (or ingestion work).
    Same,   // new
    /// An earlier evaluation, reused by a later one; no production consumer yet.
    Later,  // new
}
/// (I_κ, φ_κ): the schedule of the consumer's execution context, from its query's 𝒯_q. New (doc section 3).
pub struct EvaluationSchedule {
    /// I_κ in milliseconds.
    pub interval_ms: u64, // new
    /// φ_κ in milliseconds.
    pub phase_ms: u64,    // new
}
/// r^phys_{c,i}. New (doc section 3).
pub struct PhysicalRequirement {
    /// When the consumer runs.
    consumer_timing: ExecutionTiming,           // new
    /// Which evaluation's output it reads.
    reads_from: Evaluation,                   // new
    /// The schedule of the consumer's context; None until #1200 PR 2.
    consumer_schedule: Option<EvaluationSchedule>,  // new
}
impl PhysicalRequirement {
    /// Rejects (IngestionTime, Later).
    pub fn new(consumer_timing: ExecutionTiming, reads_from: Evaluation,
               consumer_schedule: Option<EvaluationSchedule>) -> Result<Self, InvalidPhysicalProperties>; // new
}
/// α^phys_{u,c,i}: #1200's timing and lifetime table. New; replaces readable_at.
pub fn satisfies_physical(producer: &PhysicalProperties, requirement: &PhysicalRequirement) -> bool;

crates/cost/src/execution_demand.rs, crates/cost/src/usage.rs, crates/plan-selection/src/groups.rs, crates/plan-selection/src/report.rs

/// How often a materialized output is computed. New.
pub enum MaterializedRunRate {
    /// Once per second of arriving data, at ingestion time.
    PerSecondOfArrivingData, // new
    /// At the combined evaluation rate of the queries reaching it.
    PerEvaluation,           // new
    /// Once per request: a chain materialized across evaluations builds only its newest window.
    PerRequest,              // new
}
pub fn materialized_run_rate(properties: PhysicalProperties) -> Option<MaterializedRunRate>; // new; None = not materialized

pub fn materialized_tumbling_window_chains(dag: &PhysicalASAPDAG)
    -> Result<HashSet<PhysicalASAPNodeId>, (PhysicalASAPNodeId, AnalyticalCostError)>; // was -> HashSet<_>
pub enum AnalyticalCostError {
    /// A plan node fails the export's validation (e.g. an invalid pair): a planner bug.
    InvalidPhysicalDag(PhysicalASAPDAGValidationError), // new; replaces InvalidPhysicalProperties { node, reason }
    // InvalidPhysicalProperties { node, reason },      // removed
    // every other variant unchanged
}
// impl PhysicalASAPDAGNode { pub fn physical_properties(&self) -> Result<..>; } // removed: inlined

// Stage 3 (crate-private)
pub(crate) enum RunKind { PerContext, Ingestion, Materialized }   // Materialized was Shareable
pub(crate) enum ContextKind { Query(usize), Ingestion(IngestionPipeline), Materialized(PhysicalCandidateId) } // Materialized was Shared

Usage: no new entry point. cargo run --release -p asap-devtools --bin asap_planner -- --workload tools/dag-viewer/workloads/planner-layering/example1.json --run-until stage3 prints the same Stage 2 candidates and selected plan as on #1214. The table tests drive each consumer with all 8 states.

Major algorithm changes

None. Every state Stage 2 offers takes the same path as before in every consumer. Cost changes only for states no generated plan reaches: maintained for (IngestionTime, AcrossEvaluations(Disk)) (#1217), and (IngestionTime, None) working memory is no longer recorded as held by a query evaluation (it never was: the old arm checked !ingestion).

Before this PR

Consumers of (timing, lifetime) used wildcards, and three places re-derived "materialized chain at query time" from raw fields.

After this PR

Each consumer names every valid state; α^phys is the defined PhysicalRequirement + satisfies_physical; the run kinds and contexts carry no copy of a candidate's properties; one function classifies how often a materialized output runs.

Verification

  • Head 198b9a9f1 (on refactor(types, stages): physical properties off the logical IR (OperatorNode has no timing or lifetime) #1223 5ef723d9c): cargo fmt --all -- --check and cargo clippy --workspace --all-targets -- -D warnings clean. cargo test -p asap-types (254), -p asap-cost (46), -p asap-physical-asap-optimization (28), -p asap-executor (297), -p asap-planner (89, 8 wall-clock ignored), -p asap-plan-selection --lib (146), -p asap-devtools (109), -p asap-integration-tests (364): 0 failed. After the consumer renames: -p asap-types --lib (144) and -p asap-plan-selection --lib (146) again, 0 failed.
  • New table tests: satisfies_physical_over_every_valid_state, is_materialized_over_every_valid_state, later_evaluation_at_ingestion_is_the_invalid_requirement; exactly_the_ingestion_time_one_evaluation_pairs_are_invalid checks that new accepts exactly 8 of the 10 pairs, which every other table test enumerates inline.
  • Snapshots: only sketch_plan_snapshots/fuzz-issue903-raw-retained.json's display_name changes (label query time, shared → query time, one evaluation); devtools snapshot tests pass, so no plan, cost or candidate count changes.
  • Earlier round (at 9eae875a1): UPDATE_SNAPSHOTS=1 cargo test --release -p asap-devtools --test corpus_snapshots -- --include-ignored: 9 passed, git status clean.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K1M6br9PL4SDeVS1XpezKJ

@zzylol zzylol added cost-model Cost formulas, statistics, ranking, and selection ir Pre-ASAP or post-ASAP intermediate representation optimizer Workload optimization and plan selection refactor labels Nov 5, 2026
This was referenced Nov 5, 2026
Open
Open
@zzylol
zzylol force-pushed the refactor/1200-all-valid-states branch from 17d13c3 to e5edb06 Compare November 5, 2026 04:22
This was referenced Nov 5, 2026
Closed
Closed
@zzylol
zzylol force-pushed the refactor/1200-all-valid-states branch from e5edb06 to 9eae875 Compare November 5, 2026 17:11
@zzylol
zzylol changed the base branch from refactor/1200-timing-lifetime to refactor/1200-physical-properties-off-logical-ir November 5, 2026 17:11
zzylol and others added 2 commits November 5, 2026 20:19
@zzylol @claude
refactor: handle every valid timing x lifetime state explicitly
2854864 
PhysicalProperties stays the struct of (ExecutionTiming, Lifetime) from
or arms that list their states, and one unreachable!() arm for the invalid
(IngestionTime, OneEvaluation): new, readable_at, summary_properties_allowed,
run_kind, node_usage, materialized_tumbling_window_chains (now Result, the
error names the node), materialized_run_rate (one classification shared by
computations and the MILP's context rates) and the devtools phase_label.
RunKind::Shareable and ContextKind::Shared become Materialized (kappa_m) and
carry no copy of the candidate's properties. The selector and select path
share cost_error and materialized_chains_of; Stage 3's window_read_through
uses the shared chain predicate. Table tests over every valid state
(PhysicalProperties::every_valid) for each consumer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K1M6br9PL4SDeVS1XpezKJ
@zzylol @claude
refactor: review fixes for every valid state; PhysicalRequirement and…
d32bdc6 
… satisfies_physical

Replaces readable_at with the interfaces #1200 and the concepts doc
define: PhysicalRequirement { reader_timing, reads_from: Evaluation }
(new rejects an ingestion-time reader of a later evaluation) and
satisfies_physical(producer, requirement) for alpha^phys. is_materialized
moves from Lifetime to PhysicalProperties, as the doc defines it;
Lifetime::is_across_evaluations, now unused, is deleted.

Review fixes: the realizer checks the whole selected (timing, lifetime)
after the timing pass; ErrorSelectingPlan::InvalidCandidate and
materialized_chains_of are deleted (materialized_tumbling_window_chains
reports the node, as plan_cost does, through cost_error); the warm start
uses materialized_run_rate; shareable/shared names in Stage 3 say
materialized (kappa_m); is_query_time_materialized_tumbling_window_chain
says what it tests; the labels say "query time, one evaluation" and
"query time, materialized across evaluations"; remaining consumers match
every valid state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the refactor/1200-all-valid-states branch from 9eae875 to d32bdc6 Compare November 5, 2026 20:21
@zzylol
zzylol force-pushed the refactor/1200-physical-properties-off-logical-ir branch from 6500daf to 5ef723d Compare November 5, 2026 20:21
@zzylol zzylol changed the title refactor(types, stages): handle every valid timing × lifetime state explicitly refactor(types, stages): every valid timing × lifetime state explicit; PhysicalRequirement and satisfies_physical Nov 5, 2026
@zzylol zzylol mentioned this pull request Nov 5, 2026
Closed
@zzylol @claude
refactor: no interim abstractions in the physical properties types
198b9a9 
User decisions (2026-11-05). PhysicalRequirement is
{ consumer_timing, reads_from: Evaluation, consumer_schedule:
Option<EvaluationSchedule> }; EvaluationSchedule { interval_ms, phase_ms }
is the schedule of the consumer's context (I_kappa, phi_kappa), None until
the schedule sigma. Names this stack introduces say consumer, not reader
(materialized_consumers, materialized_consumers_first).

The five named PhysicalProperties consts and not_materialized_at are
deleted; callers build PhysicalProperties::new(timing, lifetime).
InvalidPhysicalRequirement folds into InvalidPhysicalProperties
(LaterEvaluationAtIngestion). Cost reports an invalid pair through the
export's PhysicalASAPDAGValidationError::InvalidPhysicalProperties,
wrapped in AnalyticalCostError::InvalidPhysicalDag, instead of a parallel
variant. Inlined: PhysicalASAPDAGNode::physical_properties(), cost_error,
lists_every_node_of and its debug assertion; the tumbling-window-chain
predicate is private to asap-cost. Test-only helpers inlined:
PhysicalProperties::every_valid (tests enumerate the pairs new accepts),
default_physical_properties and physical_properties_at_ingestion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol zzylol changed the title refactor(types, stages): every valid timing × lifetime state explicit; PhysicalRequirement and satisfies_physical refactor(types, stages): every valid timing × lifetime state explicit; PhysicalRequirement, satisfies_physical, no named states Nov 5, 2026
@zzylol

zzylol commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Closed: replaced by #1207's new order (interfaces first, then one PR per stage), starting with PR 0 #1226. The code on this branch is reused in PRs 1–8; the branch is kept.

@zzylol zzylol closed this Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

cost-model Cost formulas, statistics, ranking, and selection ir Pre-ASAP or post-ASAP intermediate representation optimizer Workload optimization and plan selection refactor

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@zzylol