Repository navigation
feat: refresh windows: refresh groups and a per-window ingestion charge (#1151, lands #1152–#1154 on main) - #1182
Merged
Merged
Conversation
…un (#1151) (#1153) RefreshWindow, PhysicalASAPDAG::refresh_window and ingestion_refresh_groups derive which ingestion-time views share one run; the install compile check and the stand-in runtimes refresh per group, and a counting-store test shows one raw read per window per group. No cost change. Part of #1151.
…1154) asap_cost::execution_demand::computations counts a NotMaterialized ingestion-time node once per refresh window of its MatAtIngestion consumers instead of once per consumer, and the old Stage 3 engine (general_schedule, DirectCostProfile::selected_cost, direct_cost::encode) charges it the same way. Snapshots unchanged. Closes #1151 except the backend follow-up.
… rebased onto main The refresh-window commits (#1152-#1154) were written before #1163 renamed PaneChain to SummaryTumblingWindowChain and pane_width_ms / pane_count / newest_panes to tumbling_window_width_ms / tumbling_window_count / newest_tumbling_windows; rename their uses so they build on main.
zzylol
added a commit
that referenced
this pull request
Nov 2, 2026
zzylol
added a commit
that referenced
this pull request
Nov 2, 2026
…ding (#1163, part 2) (#1176) * refactor: rename every remaining pane name to the tumbling-window wording (#1163, part 2) Helper types, functions, modules, files, test names, comments and docs: Pane* becomes TumblingWindow*, pane_* becomes tumbling_window_*, and the word "pane" becomes "tumbling window" (e.g. PaneWidthDomain -> TumblingWindowWidthDomain, pane_chain.rs -> tumbling_window_chain.rs, expand_pane_chains -> expand_tumbling_window_chains). The planning report's pane_width_domain and pane_widths_pending become tumbling_window_* too, and the DAG viewer upgrades them in older documents. The deployed wire contract keeps its names: TumblingWindowLayout still reads and writes pane_width_ms and pane_origin_ms, WindowEdgeCoverage pane_aligned, TumblingWindowCoverageError ZeroPaneWidth / UnknownPaneOrigin / pane_phase_ms, and WindowEdgeCompatibility RequiresAlignedPanePhaseOrExactBoundaryResidual. No behavior change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor: apply review of #1163 part 2 - viewer: expanded tumbling-window nodes keep one CSS class (tumblingWindow) that app.js styles, with a test; JS data fields are camelCase; - historical docs, artifact paths and plan ids keep their recorded names; - code names inside comments and labels keep identifier form; - wording: doubled words, hyphenated compound modifiers, contrasted terms; - snapshot labels updated to the new display and cost text; - test constants name what they hold (TUMBLING_WINDOW_COUNT, ...); - the coverage errors' wire names are tested; cargo fmt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test: snapshot keys in sorted order after the tumbling_window_retention_ms rename Part 1 renamed pane_retention_ms in the snapshot JSON in place; the snapshots store keys sorted, so the renamed key now sorts after proven_optimal. The regenerated snapshots equal the checked-in ones as JSON (22 of 22). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor: rename pane names that main added since part 2 started (#1163) Reruns the part 2 rename over the files main changed (#1174, #1178, #1180, #1182) and reapplies the review's wording fixes there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#1151's three PRs (#1152, #1153, #1154) were merged into
feat/1095-stage3, not main. #1144 (#1095's group MILP) builds on them through #1157. Landing them on main lets #1144 be rebased onto main and merged there, instead of keeping two long-lived branches.What
The three reviewed commits, rebased onto main, plus one fix-up commit:
docs/design_docs/physical-planning-and-deployment.md, "Refresh groups").PhysicalASAPDAG::refresh_window/ingestion_refresh_groups;plan_installable's runtime refreshes each refresh group in one ingestion run.asap_cost::execution_demand::computationscharges aNotMaterializedingestion-time node once per distinct refresh window of itsMatAtIngestionconsumers, not once per consumer.PaneChain,pane_width_ms,pane_countandnewest_panesbecomeSummaryTumblingWindowChain,tumbling_window_width_ms,tumbling_window_countandnewest_tumbling_windows, plus rustfmt.Behavior change on main: plans where one ingestion
Scanfeeds several views of the same refresh window become cheaper, because theScanis charged once per window, not once per view. So Stage 3 can select differently. #1179 generalizes this into explicit shared-scan candidates.How
See #1152–#1154 for the design and the reviews. The rebase onto main was textually clean. The only change beyond the three commits is the rename fix-up.
Key code interfaces added / modified
As in #1153 and #1154; unchanged by the rebase except for the #1163 names. Summary:
PhysicalASAPDAG::refresh_window(node) -> Option<RefreshWindow>;RefreshWindow { width_ms, phase_ms }, withRefreshWindow::of_window;ingestion_refresh_groups;asap_cost::execution_demand::computations: the per-refresh-window charge.Major algorithm changes
#1154's ingestion charge, as reviewed there. Before: one computation per
MatAtIngestionconsumer. After: one per distinct refresh window among those consumers, and a consumer with an unbounded window counts alone.Before this PR
main charged a shared ingestion
Scanonce per view, and the runtime contract did not group ingestion refreshes by refresh window.After this PR
main has refresh groups and the per-refresh-window charge, the base #1144 expects.
Verification
On the rebased branch:
cargo check --workspace --all-targets(exceptasap-frontend-metricsql, whose test target fails to compile on main itself:lowering_semantics.rs:160callslower_promql_query_workloadwith one argument);cargo fmt --all --check;cargo test -p asap-types --lib(142) and--test types -- materialized_view(11);cargo test -p asap-cost(all passed);cargo test -p asap-plan-selection --lib(285);cargo test -p asap-planner --test planner(82, 3 ignored);cargo test -p asap-integration-tests --test integration -- materialization_e2e planner_layering(88, 21 ignored).These ran on the rebase onto c83aa19. Re-rebasing onto 59e8825 (#1178, #1180) was clean; the workspace check and fmt were re-run there.