Skip to content

feat: refresh windows: refresh groups and a per-window ingestion charge (#1151, lands #1152–#1154 on main) - #1182

Merged
zzylol merged 4 commits into
mainfrom
feat/1151-refresh-windows
Nov 2, 2026
Merged

zzylol merged 4 commits into
mainfrom
feat/1151-refresh-windows

Conversation

@zzylol

@zzylol zzylol commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Why

#1151's three PRs (#1152, #1153, #1154) were merged into feat/1095-stage3, not main. #1144 (#1095's group MILP) builds on them through #1157. Landing them on main lets #1144 be rebased onto main and merged there, instead of keeping two long-lived branches.

What

The three reviewed commits, rebased onto main, plus one fix-up commit:

Behavior change on main: plans where one ingestion Scan feeds several views of the same refresh window become cheaper, because the Scan is charged once per window, not once per view. So Stage 3 can select differently. #1179 generalizes this into explicit shared-scan candidates.

How

See #1152–#1154 for the design and the reviews. The rebase onto main was textually clean. The only change beyond the three commits is the rename fix-up.

Key code interfaces added / modified

As in #1153 and #1154; unchanged by the rebase except for the #1163 names. Summary:

  • PhysicalASAPDAG::refresh_window(node) -> Option<RefreshWindow>;
  • RefreshWindow { width_ms, phase_ms }, with RefreshWindow::of_window;
  • ingestion_refresh_groups;
  • asap_cost::execution_demand::computations: the per-refresh-window charge.

Major algorithm changes

#1154's ingestion charge, as reviewed there. Before: one computation per MatAtIngestion consumer. After: one per distinct refresh window among those consumers, and a consumer with an unbounded window counts alone.

Before this PR

main charged a shared ingestion Scan once per view, and the runtime contract did not group ingestion refreshes by refresh window.

After this PR

main has refresh groups and the per-refresh-window charge, the base #1144 expects.

Verification

On the rebased branch:

  • cargo check --workspace --all-targets (except asap-frontend-metricsql, whose test target fails to compile on main itself: lowering_semantics.rs:160 calls lower_promql_query_workload with one argument);
  • cargo fmt --all --check;
  • cargo test -p asap-types --lib (142) and --test types -- materialized_view (11);
  • cargo test -p asap-cost (all passed);
  • cargo test -p asap-plan-selection --lib (285);
  • cargo test -p asap-planner --test planner (82, 3 ignored);
  • cargo test -p asap-integration-tests --test integration -- materialization_e2e planner_layering (88, 21 ignored).

These ran on the rebase onto c83aa19. Re-rebasing onto 59e8825 (#1178, #1180) was clean; the workspace check and fmt were re-run there.

…un (#1151) (#1152)

The deployment contract refreshes the ingestion-time views of one refresh window in one run per window end; the run's roots are the group's views whose state for that window is not yet stored. Part of #1151.
…un (#1151) (#1153)

RefreshWindow, PhysicalASAPDAG::refresh_window and ingestion_refresh_groups derive which ingestion-time views share one run; the install compile check and the stand-in runtimes refresh per group, and a counting-store test shows one raw read per window per group. No cost change. Part of #1151.
…1154)

asap_cost::execution_demand::computations counts a NotMaterialized ingestion-time node once per refresh window of its MatAtIngestion consumers instead of once per consumer, and the old Stage 3 engine (general_schedule, DirectCostProfile::selected_cost, direct_cost::encode) charges it the same way. Snapshots unchanged. Closes #1151 except the backend follow-up.
… rebased onto main

The refresh-window commits (#1152-#1154) were written before #1163 renamed
PaneChain to SummaryTumblingWindowChain and pane_width_ms / pane_count /
newest_panes to tumbling_window_width_ms / tumbling_window_count /
newest_tumbling_windows; rename their uses so they build on main.
@zzylol zzylol added enhancement New feature or request cost-model Cost formulas, statistics, ranking, and selection labels Nov 2, 2026
@zzylol
zzylol merged commit 611f9ed into main Nov 2, 2026
2 of 3 checks passed
zzylol added a commit that referenced this pull request Nov 2, 2026
Reruns the part 2 rename over the files main changed (#1174, #1178, #1180,
#1182) and reapplies the review's wording fixes there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol added a commit that referenced this pull request Nov 2, 2026
…ding (#1163, part 2) (#1176)

* refactor: rename every remaining pane name to the tumbling-window wording (#1163, part 2)

Helper types, functions, modules, files, test names, comments and docs: Pane*
becomes TumblingWindow*, pane_* becomes tumbling_window_*, and the word
"pane" becomes "tumbling window" (e.g. PaneWidthDomain ->
TumblingWindowWidthDomain, pane_chain.rs -> tumbling_window_chain.rs,
expand_pane_chains -> expand_tumbling_window_chains). The planning report's
pane_width_domain and pane_widths_pending become tumbling_window_* too, and
the DAG viewer upgrades them in older documents.

The deployed wire contract keeps its names: TumblingWindowLayout still reads
and writes pane_width_ms and pane_origin_ms, WindowEdgeCoverage pane_aligned,
TumblingWindowCoverageError ZeroPaneWidth / UnknownPaneOrigin / pane_phase_ms,
and WindowEdgeCompatibility RequiresAlignedPanePhaseOrExactBoundaryResidual.
No behavior change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor: apply review of #1163 part 2

- viewer: expanded tumbling-window nodes keep one CSS class (tumblingWindow)
  that app.js styles, with a test; JS data fields are camelCase;
- historical docs, artifact paths and plan ids keep their recorded names;
- code names inside comments and labels keep identifier form;
- wording: doubled words, hyphenated compound modifiers, contrasted terms;
- snapshot labels updated to the new display and cost text;
- test constants name what they hold (TUMBLING_WINDOW_COUNT, ...);
- the coverage errors' wire names are tested; cargo fmt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: snapshot keys in sorted order after the tumbling_window_retention_ms rename

Part 1 renamed pane_retention_ms in the snapshot JSON in place; the snapshots
store keys sorted, so the renamed key now sorts after proven_optimal. The
regenerated snapshots equal the checked-in ones as JSON (22 of 22).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor: rename pane names that main added since part 2 started (#1163)

Reruns the part 2 rename over the files main changed (#1174, #1178, #1180,
#1182) and reapplies the review's wording fixes there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol deleted the feat/1151-refresh-windows branch November 3, 2026 02:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cost-model Cost formulas, statistics, ranking, and selection enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant