Repository navigation
fix(planner)!: decide each model replay from its own change only - #895
Merged
Merged
Conversation
Remove upstream backfill cascades. A model's backfill now comes only from its own change and its own replay_on_change, so a first-run, renamed or replayed upstream no longer forces a full or bounded replay of downstream incremental models. A changed function counts as an effective query change of the models that call it directly. Those models plan as function_changed with the cause named and apply their own replay_on_change; models further downstream continue forward. The function-level replay_on_change header is removed. A model that sets full_refresh false now refuses (S203) any planned full rebuild other than its own first run or its own query change with replay_on_change full. A FULL backfill on a non-microbatch model with cursor intrinsics reports the FULL backfill and its cause instead of an unresolved-bounds error. Plan text drops the Upstream changed group and adds a Function changed group; plan JSON drops the cascade field and upstream_changed reason and adds changed_functions to direct callers. BREAKING CHANGE: backfills no longer cascade from upstream models; each model replays only for its own change per its own replay_on_change. The function-level replay_on_change header is removed. Plan JSON drops the cascade field and the upstream_changed reason, and adds changed_functions on direct callers of changed functions and query_changed on renamed models whose query changed. New planner error S203 refuses full rebuilds of full_refresh false models not requested by their own configuration.
A recognised migrate_from rename of a table or view whose handed-over definition differs from its new query now keeps the handed-over identity and plans as renamed with a query change, instead of a first run. Its backfill comes from the model's own replay_on_change (forward-only by default). Once the renamed model has built after the recorded rename, its own fingerprint is used, so later plans no longer report the query change.
…refresh protection
SQL FUNCTION() headers and Python @udf(...) decorators now fail discovery with D002 when they use a key that functions do not support, naming the key and its file and line, the same way HOOK() and AUDIT() headers reject unsupported keys. Previously such keys were silently ignored.
Custom materializations now receive query_changed for function_changed models as well as query_changed ones, since a changed directly called function is the model's own query change.
S203 is no longer evaluated for models the plan skips because a source freshness check or an external upstream blocked them.
…m columns for replay A downstream model whose only difference is __ref and cursor_inputs names rewritten for an upstream renamed with migrate_from in this plan is no longer a query or config change: its migration fingerprint with the renamed names mapped back to their origins must equal its stored one. Views are still recreated and incremental models continue forward; a real edit alongside the rewrite is a normal query change. Schema findings inferred from a model's unchanged SQL, such as a column an upstream added through SELECT *, no longer trigger replay_on_change; they evolve forward through on_schema_change. Changes to the model's own declared columns still follow its replay_on_change.
…orded renames A downstream model is no longer a query change when its compiled SQL, with __ref names of renamed models substituted back to their origin names, hashes to its stored definition hash. Substitution touches only reference calls outside comments and string literals, so it works for any SQL, including models the migration fingerprint parser cannot normalize. The rename map combines this plan's handovers with renames recorded in migration history after the downstream's last build, so renames built in an earlier run or whose migrate_from was removed are recognized too. Migration events are read only when a selected model's query hash changed and the migration table was already listed by the warehouse snapshot. A column removed from the warehouse is attributed to the model's declared columns only when no inferred columns exist, so an upstream dropping a column a partial columns block does not declare evolves forward instead of following replay_on_change.
…inition changed A schema finding follows replay_on_change only when the model's own SQL, header config, contract, or called functions changed in this plan, with rename-rewritten references not counting as a change. When the model is unchanged, every schema difference comes from upstream, whether added, removed or retyped, declared or inferred; it evolves through on_schema_change and contract enforcement as before, so an enforced contract whose declared column disappeared upstream still fails the build instead of replaying. The interim removed-column source heuristic is reverted.
kvlonge
enabled auto-merge (squash)
October 1, 2026 12:13
…a changes Model fingerprint metadata now carries a declared_columns_hash key covering the model's columns block or schema YAML columns (name, type, nullable), dynamic column families, type enforcement and contract setting. The key is additive, sits outside version identity, and is ignored by config-change comparison. A schema finding follows replay_on_change when this hash differs from the recorded one, alongside SQL, config, contract and function changes. Rows written before the key existed have no hash, so their declared columns count as unchanged and the model continues forward once.
…model The migration fingerprint cache keyed entries on the raw fingerprint metadata JSON, so the declared_columns_hash key made build-time entries miss the value computed during planning. The cache now strips that key before keying, as the key never contributes to the fingerprint.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Planning a rename-and-republish change could replay dozens of existing incremental models from scratch. Upstream backfills cascaded downstream (a first-run upstream gave FULL to its consumers), a
migrate_fromrename whose SQL changed was planned as a first run,full_refresh falseonly gated the CLI flag, and a FULL backfill on a non-microbatch cursor model failed with a misleading error (CHI-455 items 1-4).Changes
replay_on_change.replay_on_changeheader is removed; FUNCTION and@udfheaders reject unknown keys (D002).migrate_fromrename is not a query change; schema changes inferred purely from upstream columns apply forward instead of replaying.full_refresh falsemodel fails the plan if anything other than its own first run or ownreplay_on_change fullquery change would fully rebuild it.upstream_changedreason and JSONcascaderemoved;changed_functionsandquery_changedadded.BREAKING CHANGE: no backfill cascade; function
replay_on_changeremoved; plan JSON fields changed; new S203.Verification
sqb renamedownstreams, upstream column additions, function changes, S203; plan text, JSON and built rows.make test,make check-ci; one independent review plus a follow-up.