Repository navigation
fix(incremental): reject cursor models with no inputs and allow empty cursor inputs - #965
Merged
Merged
Conversation
…e missing or empty Breaking: a cursor-based incremental model that reads no __ref(), __source() or __seed() and lists no cursor_inputs (for example one that reads a hard-coded table) is now compile error P011, pointing at its cursor declaration. This applies to every incremental strategy (delete_insert, append and merge). Such models previously re-read everything on every run because no input could bound their window. When cursor inputs have no rows: - An incremental run succeeds without changing the destination, still runs the model's audits against it, and reports "no input rows (<input>.<column>)" in plan and build output. Build JSON lists empty_cursor_inputs; plan JSON reports resolution_status "no_input_rows" with empty_inputs. - When only some inputs are empty and others have new rows, the model makes no progress and plan and build show a warning such as "waiting on empty input stg_refunds.refunded_at; other inputs have new rows". It counts toward the build summary's WARN= total and the asset's JSON warnings, and JSON adds "waiting_on_empty_inputs": true. Exit code and status are unchanged. - A first build creates the destination empty. - A rebuild that would replace an existing destination with nothing (replay_on_change full after a query change, including aliased models, or a watermark microbatch --full-refresh in any watermark mode) fails with S302 and keeps the existing rows. Models whose inputs are built earlier in the same run follow the same rules; previously such a model could advance past an empty input and skip rows that input later delivered. Inputs are named by their project names, including inputs built earlier in the same run. The window is decided before any delta or staging relation is created, and other window failures name the model, its inputs and the --start-cursor/--end-cursor flags.
kvlonge
enabled auto-merge (squash)
October 6, 2026 08:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Cursor-based incremental models had no defined behaviour with no inputs or empty cursor inputs. Empty inputs could fail late after staging work, or leave an existing table empty on a rebuild.
Changes
__ref()/__source()/__seed()and lists nocursor_inputs(for example one reading a hard-coded table) is now compile errorP011, for every incremental strategy (delete_insert,append,merge). Such models previously re-read everything on every run.no input rows (<input>.<column>). JSON:empty_cursor_inputs(build),resolution_status: "no_input_rows"andempty_inputs(plan).waiting on empty input <input>.<column>; other inputs have new rows, counted inWARN=and the JSONwarnings, withwaiting_on_empty_inputs: true. Exit code and status are unchanged.replay_on_change full, aliased models included, or watermark microbatch--full-refreshin any watermark mode) fails withS302and keeps the rows.Verification
P011, and Dagster check results on a no-op run. Text and--jsonoutputs are both checked.make test,make check-ci.