Repository navigation
Conversation
zzylol
force-pushed
the
refactor/1207-properties
branch
4 times, most recently
from
November 6, 2026 15:37
5ef1ef3 to
d9d64df
Compare
zzylol
force-pushed
the
refactor/1207-properties
branch
from
November 6, 2026 15:47
d9d64df to
2a62109
Compare
zzylol
force-pushed
the
refactor/1207-properties
branch
from
November 6, 2026 16:30
2a62109 to
b87b7c0
Compare
zzylol
changed the base branch from
refactor/1207-interfaces
to
refactor/1207-crates-per-stage
November 6, 2026 16:30
zzylol
force-pushed
the
refactor/1207-properties
branch
3 times, most recently
from
November 6, 2026 17:12
460e9c7 to
ba60db8
Compare
zzylol
force-pushed
the
refactor/1207-crates-per-stage
branch
from
November 6, 2026 18:56
4d55373 to
ae4843e
Compare
zzylol
force-pushed
the
refactor/1207-properties
branch
from
November 6, 2026 18:56
ba60db8 to
60a16ee
Compare
zzylol
force-pushed
the
refactor/1207-crates-per-stage
branch
from
November 6, 2026 19:10
ae4843e to
8d020b0
Compare
zzylol
force-pushed
the
refactor/1207-properties
branch
from
November 6, 2026 19:12
60a16ee to
107a77b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Every later stage reads the property and requirement checks of the concepts doc, section 2:
parameter_list).producer_satisfies_logical_requirements).PhysicalProperties::new) and edges withproducer_satisfies_physical_requirements).delivers_on).PR 0 defined these signatures with
todo!()bodies. This PR implements them, so that the stages built on them start from checked properties.What
PhysicalProperties::newrejects the one invalid combination, (IngestionTime, OneEvaluation).PhysicalCandidate::delivers_on(memo, context)implements the per-context alignment table: a windowedAcrossEvaluationsproducer of widthFixedIntervalAt, everyScheduledtime, or aOneTimequery's time. It is false when the times are not known (FixedIntervalwithout a phase,OneTimewithout a time,EstimatedRate,Unknown). ANoneorOneEvaluationproducer runs in the consumer's own context, so it is true for it. The ingestion and materialized contexts staytodo!()for PR 7.QueryWorkloadEntry::evaluation_grid_msmoved toQueryRecurrence::evaluation_grid_ms(&self, range_evaluation)intypes/src/workload/mod.rs, so the context can call it; the entry method now delegates (no duplicated logic). It also coversScheduled(gcd of the times) andOneTimewith a time; its only callers are the workload tests, whose cases are unchanged.producer_satisfies_physical_requirementsimplements the timing × lifetime table only.LogicalProperties::newrejects a sketch build without parameters (SketchBuildWithoutParameters), parameters on an operator without a sketch family or of another family, and parameters not in the family's list (ParametersNotInList). It also rejects a window of width 0, and a window on anything but aSummaryAggwhose states merge.parameter_listreturns each family's predefined θ list. Heap variants take each width × depth with heap sizes from the 1-2-5 series 10 … 10 000 (70 entries); KMV's and UnivMon's distinct-count entries stop at ε = 0.01.LogicalRequirements::newstores any floor of the family's parameter type, listed or not (e.g. heap_size ≥ k exactly).producer_satisfies_logical_requirementscompares θ per family on the parameters that set ε.size_paramsand its helpers) are now#[cfg(test)]. Their only remaining role is a test that keeps the predefined lists equal to them.sizing::default_size_params,test_support::default_size_paramsandtest_support::cms_widthare deleted (no caller).No other stage changes. Their
todo!()s stay.Decisions and pending items
parameter_listcontentsHydraCms,HydraCountSketch,HydraKll,HydraHll,HydraUnivMon), norHllHip,KllDynamic, … So no Hydra build is valid yet (ParametersNotInList).dominatesstill compares Hydra sizes per kind, and a unit test checks that.kand KMVk≥; DDSketchalpha≤; HLLprecision≥; CMS and CountSketchwidth,depth≥; heap variants alsoheap_size≥; UnivMon all four sizes ≥; Hydra all sizes of one kind ≥. The recordedfailure_probability,relative_erroranddistinct_failure_probabilityare not compared, because every listed configuration records δ = 0.01. Another family never satisfies.SummaryAggwhose family has a merge (FamilyCapability::of(family).merge_inputs.is_some()). ASummaryTumblingWindowChainalready carries its width, so it is rejected.parameters: NoneSketchBuildWithoutParameters). Exact operators keepparameters: None.LogicalProperties::newParametersNotInList). Because Hydra lists are empty, no Hydra build is valid until they get lists.LogicalRequirements::newchecks nothing and returnsSelf(PR 0).PhysicalCandidate::newand its grid variants are deleted with it, since it existed only for the grid checks;PhysicalCandidate's fields stay public as in the doc.AcrossEvaluationsproducer indelivers_onfalse: the doc's alignment table says it is not offered yet (it needs its own delivery rule).producer_satisfies_physical_requirementstakes nomemoMemo::candidatedelivers_onreads the layout through it. PR 2 keepsfind_or_add_groupandadd_candidate.#[cfg(test)]estimators::size_params,topk_capacity,cms::cms_width,count_sketch::count_sketch_width,ddsketch::size_params,hll::hll_precision, and univmon'ssize_params,l2_shape,MAX_LAYERS,MAX_L2_ROWS,HASH_BITS,BASELINE. Production never sizes from ε any more; the lists replace it.New abstractions not in #1207 or the doc (pending approval)
QueryRecurrence::evaluation_grid_ms(&self, range_evaluation: Option<RangeEvaluation>) -> Option<u64>(moved fromQueryWorkloadEntry::evaluation_grid_ms, which delegates)types/src/workload/mod.rsdelivers_onneeds the grid of anExecutionContext, which has the recurrence and range evaluation but no entry.family_of_operator,family_of_parameters,can_be_windowed,dominatestypes/src/stages/candidates.rsKLL_PARAMETERS…UNIVMON_PARAMETERSandconst fnskll,hll,kmv,univmon_l2,univmon_distinctparameter_listreturns.summary_agg,sketch(candidates tests);memo,candidate,evaluation,every, constantsUNWINDOWED/WINDOWS_5S/WINDOWS_10S(memo_candidates tests)How
PhysicalProperties::newrejects (IngestionTime, OneEvaluation) and accepts every other (τ, ℓ).delivers_onmatches ℓ:NoneandOneEvaluationare true (they run in the consumer's context κ,AcrossEvaluationsreads ω frommemo.candidate(logical_candidate_id).logical_properties; without a tumbling layout it is false. In an evaluation context,FixedInterval(no phase) is false; every other recurrence is true iffrecurrence.evaluation_grid_ms(range_evaluation)is known and a multiple ofproducer_satisfies_physical_requirementsmatches the producer's (τ, ℓ), withunreachable!for (IngestionTime, OneEvaluation):Alignment is not part of it: Stage 3 checks
delivers_onper context.parameter_listreturns constants. They are not computed, becauseasap-typescannot depend onasap-accuracy; the heap variants' lists are built at compile time by aconst fnfrom the CMS / CountSketch shapes and the heap series. An accuracy test keeps them equal tosize_paramsover the grid.producer_satisfies_logical_requirements: no floor, or an exact producer (parameters: None), satisfies. Otherwisedominates(θ_u, θ^min)holds, with one match arm per family pair. Different families are listed by the producer's variant, not_.Known limitations
parameter_lists, so no build of theirs is a valid candidate until their error models exist (Planner: error models and parameter lists for Hydra (and other unsized families), so Pass 1 can offer them #1228).Key code interfaces
Bodies are elided as
;. Every signature below except those marked// newis PR 0's, and only its body changed.crates/types/src/physical/properties.rscrates/types/src/stages/physical/memo_candidates.rscrates/types/src/stages/candidates.rscrates/accuracy/src/sizing.rs,crates/accuracy/src/test_support.rsUsage
No workload reaches these functions yet: Pass 1 (PR 3), Pass 2 (PR 4) and Stage 2 (PR 5) are still
todo!(). The flow below is the one those PRs will run, for the doc's running example (query A: p95 over 1 m, every 10 s). It uses the calls the table tests make at the linked commit (memo_candidates.rsandcandidates.rstests).The planner's text output does not change in this PR. It still stops at Pass 1's
todo!().Major algorithm changes
None. These are new checks behind PR 0's interfaces. The θ lists replace sizing per budget share (deleted in PR 0); PR 3 uses them.
Before this PR
Every section 2 function was
todo!()(PR 0). Today's sizing chose θ from ε inside Pass 1 (deleted in PR 0).After this PR
The property and requirement checks of section 2 run, each covered by an exhaustive table test, so PRs 2–5 can build candidates and edges on them. θ comes from a fixed list per family that matches today's sizing.
Verification
Tests added. Each states one behavior:
physical::properties::testsnew_over_every_timing_and_lifetime: all 2 timings × 3 lifetimes, both media, againstOkorOneEvaluationAtIngestion;is_materializedon each valid pair.stages::physical::memo_candidates::testsdelivers_on_follows_the_alignment_table: one case per recurrence:FixedIntervalAt(with and without a range step; phases 0 and 5 s read a 5 s chain, not a 10 s one),FixedIntervalwithout a phase,Scheduled,OneTimewith and without a time,EstimatedRate,Unknown; also non-windowedAcrossEvaluations,NoneandOneEvaluation.producer_satisfies_physical_requirements_over_every_valid_timing_and_lifetime: 5 × 5 valid states, all medium pairs.stages::candidates::testslogical_properties_new_over_every_operator_parameters_and_window: 6 operators (KLL build, Hydra CMS build, exact Sum, exact IRate, KLL chain, readout) × 5 θ (none, KLL 269, CMS, Hydra CMS, unlisted KLL 270) × 3 windows.parameter_list_holds_its_family_without_repeats.logical_requirement_new_accepts_any_floor: listed and unlisted floors, including heap_size = 37.producer_satisfies_logical_requirements_compares_the_parameters_that_set_the_error: also a top-37 floor, met by heap 50 and not by heap 20.hydra_dominance_compares_every_size_of_one_kind.asap-accuracyestimators::testsparameter_list_is_the_sizing_over_the_error_grid: the lists equalsize_paramsover the grid (distinct counts of KMV and UnivMon to ε = 0.01), recorded fields included; heap variants carry every heap size of the series.a_configuration_satisfying_a_listed_floor_is_no_less_accurate: for every family, every pair of listed θ with θ_u ⪰ θ^min, and every statistic the family answers, θ_u's error is no larger at δ ≤ 0.01. This checks that dominance on the listed configurations is sound, including UnivMon's partial order.Checks run:
cargo check --workspace --all-targets: clean.cargo clippy --workspace --all-targets: clean.cargo fmt --all -- --check: clean.Test status
cargo test -p <crate> --all-features --no-fail-fastat ba60db8. Every failure panics at a later PR'stodo!(); none reaches a PR 1 function. Rows other than asap-types and asap-accuracy are unchanged from PR 0b.DeploymentCapabilities' test here)capability.rs)capability.rs) and the weighted-frequency kernelasap_plannerbinary