Skip to content

feat: parse rule facts, types and syntax trees with SQLQuack - #1374

Closed
kvlonge wants to merge 43 commits into
mainfrom
chore/sqlquack-parser-train
Closed

kvlonge wants to merge 43 commits into
mainfrom
chore/sqlquack-parser-train

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Why

Combines five reviewed SQLQuack changes into one CI run, pinning every SQLQuack crate to its main commit edfa30b2. That pin refuses a bare __ref("…") in a Snowflake FROM, so the relation-marker helper ships with it.

Changes

Each part is merged unchanged with --no-ff:

On top:

  • Cargo.toml/Cargo.lock pin every SQLQuack crate to edfa30b2 and keep main's Windows windows-sys edges (check-build-features).
  • A doc comment is cut to one line (fensu).
  • ClickHouse spells Nullable(INT) as Nullable(Int32) and BINARY(n) as FixedString(n), and refuses bare BINARY and nested Nullable. An explicit Nullable keeps family other, as before, so INT and Nullable(INT) share a spelling but not a family.

Every part, the merges and each commit on top passed one independent review.

Verification

  • make check-ci on the tip.
  • cargo test: sqlbuild-analysis, sqlbuild-rules, sqlbuild-sqltext.
  • make compiler-differential (--goldens check): 391 projects as expected.
  • The 11 Python test files the parts change: 411 passed.
  • The varied 1,000-model statistics behind the compile-cache job match main.
  • Public-repository hygiene over the tree and every commit.

DuckShop: covered per part; no new shape from combining them.

A native lint response can report that the engine does not parse the project's
dialect yet. Each such file now gets one L007 warning carrying the engine's
reason, instead of looking lint-clean. A non-string reason is rejected as an
invalid response.
…le and rules run

compile and rules run kept only selected Rule codes, so an unavailable dialect
looked clean there. L007 now passes through as a warning-severity Rule
diagnostic in compile and is left out of rules run's failing-finding count.
counts_toward_exit owns the exemption.
The plan and build rules gate failed on any finding, so an L007-only project
could not plan. Rule exceptions also validate every finding's code as a rule
code and rejected L007 with an invalid-lifecycle error. The gate now counts only
findings that count toward exit, and L007 passes through rule exceptions
unsuppressed. An end-to-end test replays a cached finding as L007 and as a rule
code through compile, rules run, plan and build.
Local scenario capture no longer asks Polyglot to read warehouse column
types. Postgres types go through sqlquack-syntax-postgres's type parser,
and Snowflake and BigQuery types through their SQLQuack cast-target
parse, behind one native function (parse_warehouse_type). DuckDB and
MotherDuck spellings pass through unchanged. Databricks nested types map
recursively to DuckDB's LIST, MAP and STRUCT, and a nested spelling that
cannot be mapped is refused with a named message instead of falling back
to JSON. Other types keep their integer width or fall back to VARCHAR.
SQL Server has no SQLQuack type parser, so its information_schema names
reached the generic fallback, which held datetime2, smalldatetime,
datetimeoffset and uniqueidentifier as VARCHAR. A SQL Server pre-mapper
now maps them to TIMESTAMP, TIMESTAMPTZ and UUID, bit to BOOLEAN, money
and smallmoney to exact decimals, and SQL Server's 8-byte float to
DOUBLE.

The fallback keeps 4-byte floats (FLOAT, FLOAT4, REAL) as FLOAT instead
of widening them to DOUBLE. A panic in the warehouse type parser now
surfaces as the ValueError capture already falls back on.
…d-local-sql

# Conflicts:
#	crates/sqlbuild-analysis/src/lib.rs
Read each type with SQLQuack's parser for the dialect, or SQLBuild's own
type reader where a dialect has none, and spell it the way the
Polyglot-based normalization did so the same aliases stay equal.
normalize_type now raises only ValueError, for an unknown dialect.
MotherDuck now reads and spells types as DuckDB. MySQL and the other
SQL analysis dialects no adapter ships raise Unknown dialect instead of
reading as generic SQL. Under ClickHouse the type reader keeps nested
type arguments, enum members with values, and named tuple fields.
sqlquack main 19c51fa2 carries the Postgres parser work and the train that
followed a9dde6a0. sqlquack-syntax-postgres now depends on stacker, which
re-resolves a few Windows-only windows-sys edges in the lock.
SQLQuack's next commit refuses a bare call in a Snowflake FROM clause,
which rejects the __ref and __source placeholders that compiled model
SQL keeps, so custom-rule syntax trees fail on Snowflake models. This
rev keeps the BigQuery parser the warehouse type parsers need.
Compiled query_sql keeps relation markers such as __ref("orders"). Snowflake
does not accept a bare function call in FROM, so newer SQLQuack parsers refuse
every Snowflake model that references another.

A shared helper in sqlbuild-analysis (relation_markers::markers_as_names) now
writes each marker call in code as a short indexed name, padded so all later
text keeps its line and column, and returns the calls by index. sql_tree reads
SQL through it for every dialect. The rules crate's dialect lexical syntax moves
to sqlbuild-sqltext so both use one mapping.
Conflict resolution: Cargo.toml and Cargo.lock pin every SQLQuack crate
(sqlquack-syntax, sqlquack-syntax-bigquery, sqlquack-syntax-postgres) to
SQLQuack main edfa30b26a0bab86661f96490b86a4f24b92db0a, replacing both the
98eefae6 and 19c51fa2 pins.
Adding stacker through the SQLQuack crates re-resolved errno, rustix,
tempfile, junction, quinn-udp and winapi-util to older windows-sys
releases. On Windows the extension build then compiled windows-sys 0.52
with features the sqb build lacks. The lockfile keeps main's edges, so
both builds share one windows-sys again.
Under ClickHouse, Nullable(T) is read as a wrapper, so an alias inside
it is respelled: Nullable(INT) is Nullable(Int32), as bare INT is.
BINARY(n) spells as FixedString(n). Bare BINARY, and a Nullable that
wraps another Nullable or a nested type, are refused as the engine
refuses them.
The rules engine stubs __table_fn calls before parsing a model. SQLQuack's
Snowflake grammar, like Snowflake, accepts a table function in FROM only
inside TABLE( ), which is how SQLBuild renders the call, so a bare stub made
every built-in Rule report the model unevaluated. Wrap the stub the same way
unless the model already wrote TABLE( ), and assert in the Snowflake
evaluation tests that the model was actually evaluated.
…rule trees

Custom Rules parse a model through the rule tree, which renames each
relation marker to a placeholder name first. SQLQuack's Snowflake grammar
refuses the bare table function call that leaves, so a custom Rule on a
Snowflake model with a table function failed to parse it. Wrap those calls
in TABLE( ) over the marker's own padding and a following space, so node
positions keep their line and column, and share the TABLE( ) detection with
the built-in Rules' stubbing.
When no space or tab follows a Snowflake table function call, the rule tree
inserts its closing parenthesis. Write the TABLE( ) wrapper over one more
padding space in that case, so text after the call keeps its column.

kvlonge commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Folded into the single combined sqlbuild PR (D625/D626), which the SQLBuild lane is assembling now and opens once #1377 lands. Branch kept.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant