Skip to content

feat(sql-macros): parse SQL macros and SQL-test planning with the native parser - #1371

Closed
kvlonge wants to merge 9 commits into
feat/analysis-route-nativefrom
feat/sql-macros-native
Closed

kvlonge wants to merge 9 commits into
feat/analysis-route-nativefrom
feat/sql-macros-native

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Draft. Waits for #1369, #1370, the DuckDB/Snowflake/SQLite arms and a SQLQuack main pin with typed markers. b666553 and 07e1f73 are validation only and get dropped on rebase. ec20510 is #1370.

Why

SQL macros and SQL-test planning parse with Polyglot, which is being removed. They now use the project's native parser, chosen by analysis_route. A dialect without a native parser reports the feature as unavailable.

Changes

  • Macro bodies parse once with the project's parser. Declarations and USING slot reads are masked as names of the same length. A type-only body fills a cast's type.
  • Probes type through NativeDialect::analyse, and type findings are B210–B219.
  • Planning reads __ref-style markers through fix(rules): read relation markers as tables in custom-rule syntax trees #1370's markers_as_names.
  • Polyglot output typing and the simple-template fast path are removed.
  • Strict xfails: 6 returns cases on arm-less adapters, 3 SQL Server planner cases, 2 list-body cases.
  • Differences from Polyglot: a partly-known body type gives no P024 (no corpus macro hits this); SQLite reads marker type text with its own grammar; SQLQuack's T-SQL parser accepts well-formed typed markers, where the engine gives 195. That last one is deliberate and isn't parity.

Verification

312 Rust tests pass; clippy, fmt, --locked and fensu are clean. DuckShop compiles 2,431 identical files. The macro corpus has 0 newly refused on DuckDB and Snowflake.

…n the native trait

Not for merging. Puts the three arms that the SQL macro and SQL-test work runs on into the native route, and pins SQLQuack to a train head that carries the native parsers' type markers. Dropped when this branch is rebased onto a main that has the arms and a SQLQuack main pin.
…ive parser

SQL macro bodies and SQL-test planning parsed with Polyglot. Both now
parse with the native parser that analysis_route gives the project's SQL
analysis dialect, and a dialect with no native parser reports the feature
as unavailable rather than falling back.

- Macro bodies parse with the project's parser instead of trying eight
  Polyglot dialects. Declarations and USING slot reads are masked as
  names of the same length, so spans line up. A body that only names a
  type is read as a cast's type.
- Macro probes type through NativeDialect::analyse over stand-in
  relations; returns, ANY-family, argument and slot checks read its
  output types, and type findings are SQLBuild's B210-B219.
- SQL-test planning parses with sql_test_parser(dialect). The planning
  template reads relation marker calls as plain names, so Snowflake
  accepts them in FROM, and a list after IN is read as a list.
- Polyglot's output-typing entry points, used only by SQL macros, and the
  token-based simple-template fast path are removed.
Compiled query_sql keeps relation markers such as __ref("orders"). Snowflake
does not accept a bare function call in FROM, so newer SQLQuack parsers refuse
every Snowflake model that references another.

A shared helper in sqlbuild-analysis (relation_markers::markers_as_names) now
writes each marker call in code as a short indexed name, padded so all later
text keeps its line and column, and returns the calls by index. sql_tree reads
SQL through it for every dialect. The rules crate's dialect lexical syntax moves
to sqlbuild-sqltext so both use one mapping.
…d markers_as_names

The planning template now names marker calls with relation_markers::markers_as_names, which sql_tree also uses, and reads them back by its MARKER_NAME_PREFIX. The SQL-test patterns no longer carry their own marker regex.
… rules

Not for merging, with the validation arms. Their calls satisfy the normalize_text_type exception, so it goes, and two comments take fensu's shape.
… native parser

Expansion cases for an adapter whose dialect has no native parser were filtered out, so they checked nothing. They now assert the P015 that says SQL macros are unavailable, and run as written once the adapter's arm lands.
…drop partly-known returns checks

Macros and SQL tests now read SQL only with the adapter's native parser, so the pages say what happens without one. Native analysis does not type list literals, so a list body trusts returns rather than failing P024 by shape, and the BigQuery NUMERIC example goes until BigQuery has an arm.

kvlonge commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Superseded: its shippable commits are in wip/native-cutover (six) and #1374 (the relation-marker fix); the two remaining commits are validation-only scaffolding marked not for merging. The branch is kept.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant