Skip to content

fix(test): resolve mocked and expected-row references consistently in SQL tests - #1023

Merged
kvlonge merged 4 commits into
mainfrom
fix/assertion-mock-references
Oct 8, 2026
Merged

kvlonge merged 4 commits into
mainfrom
fix/assertion-mock-references

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Why

In SQL unit tests a mocked name did not always mean the mock: an __assert__ calling __ref("items") on a mocked items ran the real model and made it a target. Calls in __expected__ CTEs stayed raw and failed in the warehouse. The native planner trusted optional helper targets from Python. Tests reading only mocks were skipped silently.

Changes

  • Fix (behaviour change): a mocked name means the mock in every SQL-test CTE. A mocked model is not run for an assertion and is not a target, so sqb test --select <mocked model> no longer selects the test.
  • References in __expected__ CTEs resolve like in assertions; an unmocked __ref() there runs that model.
  • Located P013 at compile for unmocked __source()/__seed()/__dbt_ref() in expected rows, assertions and read helpers, unknown models in expected rows, and model tests that mock the model they test (help: mock its inputs instead). Macro, UDF, table-function tests and scenarios are unaffected.
  • One rule for which helpers a test reads: the compiler's parsed reads. The native planner requires them with the reference targets, fails if absent, and leaves unread helpers that call references out of the test query. A reference it still cannot resolve is a located P013.
  • Simplified the helper CTE merge; docs and skill copy updated.

Verification

  • Rust tests calling the native planner directly, including the missing-reads failure
  • DuckDB E2E on python, native and native-preview, with and without sql_analysis, incl. helpers named like columns and selection; three failure-corpus cases
  • cargo test --workspace --all-features, make compiler-differential, make compiler-differential-shipped, tests/e2e/scripts/compiler_differential, make check-ci, make test

… SQL tests

A mocked name now always means the mock in every SQL-test CTE: an assertion
calling __ref() on a mocked model reads the mock instead of running the real
model, and mocked models no longer count as test targets. References written
directly in __expected__ CTEs are resolved like assertions and helpers.

Compile reports unmocked __source(), __seed() and __dbt_ref() calls and unknown
models in expected rows with P013. The native planner computes the models
helpers and expected rows read itself and rejects any reference left raw.
A model-mode SQL test whose expected rows and assertions read only mocked names
runs no model and used to be skipped silently. Compile now reports it with a
located P013 diagnostic showing how to call __ref() on the model under test.
Macro, UDF and table-function tests and scenarios keep their own targets.
…y mock-only errors

The native planner now takes the helpers a test reads and the models its
expected rows and helpers reference from the compiler, fails when a request
omits them, and leaves unread helpers that call references out of the test
query, so a helper named like a column no longer runs or fails a test. A
reference the planner still cannot resolve is a located P013 instead of an
unlocated P001. The error for a test that mocks the model it tests now
explains how to mock the model's inputs instead.
@kvlonge
kvlonge force-pushed the fix/assertion-mock-references branch from 602b480 to 82c25be Compare October 8, 2026 07:46
@kvlonge
kvlonge enabled auto-merge (squash) October 8, 2026 07:52
@kvlonge
kvlonge merged commit 7e908ac into main Oct 8, 2026
39 checks passed
@kvlonge
kvlonge deleted the fix/assertion-mock-references branch October 8, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant