Repository navigation
fix!: correctness, adapter, rules and parser bug batch - #1252
Merged
Merged
Conversation
A build whose Python check failed exited non-zero but printed "Completed successfully", because the completion footer counted SQL results and Python nodes but not Python check results. Failed checks now count as FAIL and make the line "Completed with errors", warned checks count as WARN, passing checks as PASS, and failed checks are listed under the footer's failures. The check docs also called ctx.fail, ctx.warn and ctx.pass_ with a positional message, which raises TypeError because the message is keyword-only; the examples now pass message=.
SQL-test planning collects the relation markers a query reads to swap
in their mocks. The walk skipped the left side of a parenthesised join,
so a model reading FROM (__ref("orders") AS o JOIN __ref("customers")
AS c ON ...) failed with "references __ref(\"orders\") which has no
mock" although the test mocked it. It also skipped the relations of
UPDATE ... FROM. Both are collected now, and the JSON oracle the walk is
tested against covers them.
A dict key built from a nested macro call, such as
@py.show({@py.items(): 2}) where items() returns a list, crashed
compilation with a raw TypeError traceback. It now fails with the
located P001 macro argument error, pointing at the nested call whose
value cannot be hashed and naming its type.
sqb mv refused any move of a model whose name appeared in project Python, such as @check(depends_on=model("orders")), although a move to a folder or to a file of the same name keeps the model's name, so those references stay valid. The Python name search now runs only for renames: sqb rename, and sqb mv to a file with a new name.
SQLite stores timestamps as ISO-8601 text, and its cursor windows
already compare cursors as normalised text, but compile rejected a
cursor_type timestamp column of type TEXT, such as
strftime('%Y-%m-%d 00:00:00', ordered_at), with B301. On SQLite a text
column is now a valid timestamp cursor; other adapters still reject
it.
On SQLite, CAST(x AS TIMESTAMP), DATE, DATETIME or TIME takes NUMERIC affinity and keeps only the leading number, so ISO-8601 text such as '2026-01-01 10:00:00' becomes 2026. Compile now reports a located B104 warning at each such cast in model SQL, with help to keep the text or use datetime(), date() or strftime(). Other adapters are unaffected.
sqb dbt run --json and sqb dbt build --json accepted the flag but wrote no JSON document: stdout held SQLBuild result rows and dbt output, so a caller parsing it failed. Both commands now reject --json with C238 before dbt starts and point at sqb dbt plan --json, the command that has a JSON document. BREAKING CHANGE: sqb dbt run and sqb dbt build fail with C238 when given --json, instead of ignoring it.
Local scenario replay inserted VARIANT fixture values as Python
objects, which DuckDB stores as variants of different member types
than a JSON parse gives, so an expected '{"qty": 2}'::JSON::VARIANT
never matched the replayed row and strict replay failed with X506. The
loader now loads each VARIANT value through CAST(CAST(? AS JSON) AS
VARIANT): DuckDB captures are serialised to JSON, and a string captured
from another warehouse, whose driver returns semi-structured values as
JSON text, is used as that text when it parses.
…lumns On Snowflake a star expands to the upstream's columns as Snowflake names them, upper case, so SELECT o.* under a contract declaring order_id and status reported K005 for ORDER_ID and STATUS although Snowflake treats the unquoted names as the same columns. On Snowflake a declared column now also names the upper-case form of its name; an undeclared column a star adds is still reported.
sqb load ran Python loaders on SQLite, but dlt and ingestr integration loaders failed with "Adapter 'sqlite' does not support ... loaders". dlt now writes through its sqlalchemy destination into the project's database file, dataset main unless the source names one, and ingestr gets a sqlite:/// destination URI for the same file.
…nctions sqb test copies each SQL function a test calls into the session's pg_temp schema. Behind a transaction-mode pooler a later statement can run in a session without the copy and fail. The PostgreSQL connection option temporary_test_functions = false stops the copies, so tests call the built functions as before.
sqb test registered a project's DuckDB Python UDFs on its first test connection only. Connections to one database file share them, but an in-memory database is separate per connection, so tests on the other connections failed calling the UDF. The UDFs are now registered on every test connection, including replacement connections; a connection that already sees the function keeps it.
…ject directory A DuckDB connection's attach entries failed on a command's second connection with "Unique file handle conflict ... already attached", because connections to one project database share the attachment, and a relative attach path resolved from the working directory, so running sqb from outside the project created or read another file. Attach now uses ATTACH IF NOT EXISTS, and a relative DuckDB or SQLite attach path resolves from the project directory, like database. BREAKING CHANGE: a relative DuckDB attach path now resolves from the project directory instead of the current working directory.
The planner read cursor bounds with CAST(MAX(cursor) AS VARCHAR). On SQL Server that cast renders a datetime as 'Apr 4 2026 2:30PM', so the second, incremental build of a timestamp-cursor model failed with C001 "invalid timestamp cursor value". Adapters now render the bound as text through render_cursor_text, and SQL Server converts timestamp bounds with CONVERT(VARCHAR(33), ..., 126). The SQL Server waffle-shop E2E, which builds twice, covers the incremental path.
…one is set DatabricksAdapter.schema_exists returned False whenever the target had no catalog, so sqb review's post-drop check and the janitor's leftover-schema filter always saw such schemas as missing. Without a catalog it now queries the session's current catalog through the unqualified information_schema.schemata.
…ares a line Model rule findings counted columns from the query's own text. When the query began on the MODEL header's line, as in MODEL (...); WITH ... SELECT * FROM o, SQBRMODEL102 pointed at column 35 instead of the star's file column. A finding on the query's first content line is now shifted by the characters before the query on that file line; findings on later lines and on queries below the header are unchanged.
ClickHouse and Polyglot's ClickHouse tokenizer read # and // as line comments, but SQLBuild's ClickHouse lexical syntax and the rules tokenizer listed only --. An apostrophe in such a comment then opened a string to SQLBuild's scanners, so compile and sqb rules failed with P001 "SQL reference contains an unclosed quoted string". Both tables now list --, # and //.
A plain maturin develop installs the pinned sqlbuild-cli from the package index instead of the workspace crate, so the sqb executable is a released build, and the install fails between a release commit and its publication. The contributor guide now says to always pass --skip-install or use make dev-native or make fast-native.
make skills, which regenerates the repository's bundled skill files, also ran sqb skills --global --target opencode and so wrote this checkout's development skill into the user's real OpenCode skills directory. It now regenerates only the repository files; the global install is the explicit make install-global-skill target.
Update Polyglot to a revision that fixes several analysis gaps: - a star over a derived table expands to its columns, so SELECT * FROM (SELECT ...) [AS s] no longer reports column_count 0; - star EXCLUDE, REPLACE and RENAME apply when stars expand, so an EXCLUDE no longer causes a false B216 between set operation arms; - column-alias lists on table functions and tables name their columns; - Snowflake typed lambda parameters, (total DOUBLE, item VARIANT) ->, and a subscript before a colon path, f[0]:name, parse; - Snowflake reports CONNECT_BY_ISLEAF and CONNECT_BY_ISCYCLE, which it does not support, as unknown columns; - ClickHouse DATE_TRUNC keeps its date or timestamp type instead of text, so a DATE_TRUNC timestamp cursor no longer fails B301; - BigQuery RANGE<DATE>, RANGE<DATETIME> and RANGE<TIMESTAMP> parse; - truncated T-SQL such as SELECT IF(region > 1, re is a parse error instead of a parser panic, and nested set operations keep their grouping when transpiled to SQLite. A new compile E2E covers the dialect cases, and the tests that used the T-SQL panic now assert its parse error.
…der check fails The scenario pipeline orchestration test has failed on CI with two unexpected statement events between the connection operation and the scenario attempts, and it does not reproduce locally. Its event-order assertion now reports each event's invocation, run, adapter, purpose and statement kind, so the next failure shows whether the statements came from this test's pipeline or leaked from another test.
…hangs The four-concurrent-builds Recall test once timed out after 600 s inside sqb recall query under heavy load, and the hang left no evidence. The Recall test helpers now run the CLI with PYTHONFAULTHANDLER, and a CLI run, or a concurrent build, still running after 600 s is aborted with SIGABRT so the failure carries every thread's stack.
…rtifacts A warm compile has intermittently re-planned SQL tests on CI, printing an extra SQL test planning line, and the miss left no evidence of its cause. When the SQL test artifact cache holds records but some tests cannot reuse theirs, compile now logs one debug event giving the count per reason: not recorded, identity changed, unsafe path, artifact missing, size changed, unreadable mtime or content changed. The event goes to the invocation's diagnostic log and to --debug output.
…ns type With Polyglot applying star REPLACE during expansion, a SQL macro call\nin SELECT * REPLACE (...) now takes the macro's returns type, so the\nstrict xfail that recorded the gap passed and failed the suite.
…on expansion Model rules located findings in the expanded query, so every macro\ncall that expanded into more lines than it was written in moved the\nfindings below it down; SQBRCONTRACT105 for a model with several\nmulti-line macro calls pointed many lines past the flagged output.\nCompiled model facts now keep the interpolation, declaration and macro\nexpansion passes, and rule positions map through them to the authored\nfile. A position inside an expansion points at the call that produced\nit. When the passes do not fit the query, positions fall back to the\nprevious query-relative mapping.
…indow check fails The completion-write failure-window test has failed intermittently and\ndoes not reproduce locally (36 of 36 runs pass at 12-way concurrency).\nIts checks after the failed build now report the microbatch state rows\nand that build's output, so the next failure shows which window\ndiverged.
…ma lookup The dbt wrapper case passed --json to sqb dbt run, which now rejects it, so it covers only the stripped --verbose flag and stdout progress. The metadata-SQL ownership guard now allows the Databricks constants module, which holds the information_schema name the no-catalog schema lookup uses.
A Snowflake capture stores a VARIANT as Snowflake's pretty-printed JSON text in its own key order, and local replay transpiles an expected PARSE_JSON to a DuckDB JSON value. DuckDB resolves a VARIANT column set against a JSON one to JSON and compares the text, so equal objects with other key order or whitespace failed strict replay with X506. The expected comparison now casts each column the actual DuckDB relation stores as VARIANT to VARIANT on both sides, where objects compare by key and value; arrays, value types and missing keys still differ.
Comparing replayed VARIANT columns as VARIANT values still depended on
how each side's VARIANT was built. Re-encoding DuckDB captures as JSON
changed the member types of cast-built values such as
{'qty': 2}::VARIANT or 2::VARIANT, so their scenarios failed X506 though
the values were equal.
The expected comparison now compares each column the actual DuckDB
relation stores as VARIANT or JSON through one canonical JSON text on
both sides: sorted keys, no whitespace and numbers in their shortest
exact form. A Python function on the replay connection computes it. Equal
values match however they were built, whether cast-built, JSON-built or
captured as Snowflake's pretty-printed text in its own key order; arrays,
value types and missing keys still differ. DuckDB captures load their
values natively again; another warehouse's JSON text still loads through
JSON.
Anchoring relative DuckDB file paths to the project directory also anchored ~/archive.duckdb, which DuckDB expands from HOME, so it opened <project>/~/archive.duckdb and failed. A path that is ~ or starts with ~/ now resolves from HOME before anchoring, and only other relative paths are joined to the project directory. The DuckDB docs also say that attach entries from a dbt profile resolve from the SQLBuild project directory.
Update Polyglot to a revision that reports WITH v AS (VALUES ...) as a syntax error on Snowflake, which rejects it, instead of accepting the model until it ran. Compile now fails such a model with P001 and a message that says to select from the list; VALUES in FROM stays valid.
A quoted output column such as o.payload AS "Payload" reached the comparison with its quotes, so it never matched the actual relation's VARIANT column, skipped the JSON-value comparison and failed X506. The column's unquoted name is now matched. The scenario docs now say that local replay compares VARIANT and JSON columns by JSON value, so values of non-JSON types with the same JSON text compare equal.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Correctness, adapter, rules and tooling bugs, each first reproduced by a test failing on main; parser gaps fixed in Polyglot.
Changes
sqb mvof Python-named models.~paths expand from HOME.temporary_test_functions; SQL Server ISO cursors; Databricksschema_existswithout a catalog.#///comments.f[0]:x; ClickHouse DATE_TRUNC types; BigQuery RANGE; no T-SQL panic.make skillsstays out of HOME; flaky-test and cache-miss diagnostics.BREAKING CHANGE:
sqb dbt run --jsonandsqb dbt build --jsonfail with C238; usesqb dbt plan --json.BREAKING CHANGE: relative DuckDB/SQLite attach paths, including attach entries from a dbt profile, resolve from the SQLBuild project directory.
BREAKING CHANGE: Snowflake now rejects
CONNECT_BY_ISLEAF/CONNECT_BY_ISCYCLE(E201) and bareVALUESCTE bodies, including SQL-test fixtures (P001); Snowflake supports neither.Verification
make test,make check-ci,make compiler-differentialand Polyglot suites pass.BEGIN_COMMIT_OVERRIDE
fix!: reject --json on sqb dbt run and sqb dbt build
BREAKING CHANGE: sqb dbt run --json and sqb dbt build --json now fail with C238; use sqb dbt plan --json instead.
fix!: resolve relative DuckDB and SQLite attach paths from the project directory
BREAKING CHANGE: relative DuckDB and SQLite attach paths, including attach entries from a dbt profile, now resolve from the SQLBuild project directory instead of the working directory. Paths starting with ~/ resolve from HOME.
fix!: reject Snowflake constructs Snowflake does not support
BREAKING CHANGE: on Snowflake, CONNECT_BY_ISLEAF and CONNECT_BY_ISCYCLE now raise E201, and a bare VALUES list as a CTE body, including in SQL-test fixtures, now fails with P001.
fix: count Python checks in the build completion line, find mocked references on both sides of parenthesised joins, and support sqb mv with Python references
fix: compare scenario VARIANT and JSON columns by JSON value in local replay
fix: match Snowflake star outputs against lowercase contract columns
fix: SQLite text timestamp cursors, temporal cast warnings, and dlt and ingestr loading
fix: register DuckDB UDFs on every test connection and attach each database once
feat: add a Postgres option to turn off temporary test functions
fix: SQL Server ISO cursor bounds, Databricks schema checks without a catalog, and ClickHouse line comments
fix: map rule positions back through macro expansion
fix: Polyglot star expansion with EXCLUDE, REPLACE and RENAME, column alias lists, Snowflake typed lambdas and subscript paths, nested set operation grouping, ClickHouse DATE_TRUNC and BigQuery RANGE types
fix: keep make skills out of HOME
END_COMMIT_OVERRIDE