Skip to content

fix: deferral, refactor, microbatch progress, scenario, diff and clone fixes - #1212

Merged
kvlonge merged 13 commits into
mainfrom
fix/bug-batch-2
Oct 22, 2026
Merged

kvlonge merged 13 commits into
mainfrom
fix/bug-batch-2

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Why

A batch of user-facing bugs: deferral picked the wrong origin relation for models with migrate_from, sqb mv refused harmless same-name moves, non-TTY microbatch builds printed no per-batch progress, S504 did not name the model that caused it, unkeyed diffs showed no example rows, sqb clone had no phase timings, and dbt builds dropped the live row for error-level model results.

Changes

  • Deferral resolves an unselected renamed model to its pre-migration relation in its own deferred schema and adds a plan warning. A schema that cannot be listed falls back to the current name with a warning.
  • sqb mv allows a same-name move with a pending migrate_from unless the move changes the relation.
  • Microbatch builds print one permanent line per batch when stdout is not a terminal. Query diagnostics are capped per resource.
  • S504 names the boundary model and its unmocked inputs, and lists at most five missing sources. sqb scenario test ends with a failures list.
  • Unkeyed diffs show a bounded set of example side-only rows (from_only_rows/to_only_rows in JSON).
  • sqb clone logs per-phase timings under --debug and adds phase_timings to its JSON document.
  • dbt build/run streams a result row for error-level model results and keeps warn/error result messages.
  • Regression coverage for a partial mock read by two real models, and a SQL Server test helper path fix.

Residual: sqb compile --defer-to (static path) still renders the current name of a renamed upstream, while sqb build reads the pre-migration relation.

Verification

  • New DuckDB E2E and integration tests through the real CLI and planner for each behaviour fix, plus focused unit and Rust tests.
  • Focused runs: new E2E files, clone, diff and deferral tests, dbt failing-model and deferral E2E.
  • uv sync --all-extras and make check-ci pass.

@kvlonge
kvlonge enabled auto-merge (squash) October 22, 2026 03:36
When --defer-to resolves an unselected model that the deferred target has
not built under its current name, and the model declares migrate_from, read
the migrate_from relation there instead and say so in the plan.
…move

A move that keeps the model's name and relation carries its undeployed
migrate_from with the file. Renames, and moves that change the relation,
are still refused until the earlier migration is deployed.
Captured and debug output now gets one permanent line per completed batch
with its position, cursor window, rows and elapsed time; terminals keep the
live spinner. Verbose query diagnostics are capped per resource instead of
per run, so one model's statements no longer hide another's.
Each missing column is padded once whichever real models read the mock and
however they spell it. The duplicate padding was fixed when required fixture
columns started being keyed by casefolded name; this pins the reported shape.
…t the end

When a scenario mocks some of a model's inputs but leaves others that read
sources real, S504 names that model and its unmocked inputs with the CTEs to
add, and counts the sources instead of listing one line each. sqb scenario
test now ends with the failed scenarios and their first error line.
Unkeyed diffs now sample the complete rows one side holds more often than
the other, bounded by --max-row-only-examples, in the text output and as
examples.from_only_rows and to_only_rows in --json. Key example lists stay
empty, so truncated_by_count reflects the rows actually collected.
The helper rewrote macros/time.py, which the fixture no longer has, so it
failed before reaching the warehouse. Use models/marts/_macros/time.py and
fail loudly when any fixture rewrite no longer matches.
Clone logs the duration of each target compile, source inspection, planning
and execution under --debug, and the --json-output document reports them in
seconds under phase_timings.
An error-level LogModelResult carries a message, so it was buffered as a node
message instead of reporting the node's result. Result events now take
precedence, and the row streams as soon as dbt reports it.
…wn schema

A bare migrate_from name now resolves in the model's deferred database and
schema, and the current-name check lists that schema instead of matching a
same-named relation in any deferred schema. Origins are looked up only for
unselected upstreams, after selection, and a schema that cannot be listed
keeps the previous deferral with a plan warning instead of failing planning.
The sample query now ends in LIMIT (TOP on SQL Server), so client-side
cursors no longer download every differing row, and the cursor is closed
after the fetch.
A warn or error-level dbt result event now carries its own message on the
streamed row, so test, seed and batch rows keep lines such as FAIL 3.
@kvlonge
kvlonge merged commit 78b3adc into main Oct 22, 2026
46 checks passed
@kvlonge
kvlonge deleted the fix/bug-batch-2 branch October 22, 2026 05:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant