Skip to content

Choose the planner proofs' parser through the analysis route - #1362

Closed
kvlonge wants to merge 1 commit into
wip/analysis-route-clickhousefrom
wip/planner-proofs-native
Closed

kvlonge wants to merge 1 commit into
wip/analysis-route-clickhousefrom
wip/planner-proofs-native

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Draft, stacked: the base is wip/analysis-route-clickhouse, the ClickHouse analysis-route arm, which is not for merging on its own. This PR moves onto the analysis route's follow-up branch once that branch exists (it carries NativeDialect::parse, an Unavailable route and a short re-export).

Why

The planner's proofs parse SQL with Polyglot directly, outside model analysis:

  • migration query shapes and rename fingerprints;
  • raw diff query checks;
  • relation names;
  • fixture star dependencies;
  • Snowflake's metadata-only CREATE check;
  • cursor-type classification.

To remove Polyglot, each proof has to pick its parser the same way model analysis does, through analysis_route. Each proof also has to fail safe: SQL the dialect's parser can't read gives no proof, and the planner takes its conservative path. A check is never skipped.

Changes

  • NativeDialect gains parse, which returns every statement with the parser's named gaps. read_whole becomes a provided method over parse: one statement, no gap, and not a lone unparsed Command. The ClickHouse arm implements only parse.
  • Each Polyglot caller now matches on analysis_route(dialect). On a native route it reads SQLQuack's tree. On the legacy route the Polyglot code is unchanged.
Proof Native route SQL the native parser can't read
Migration query shape (query_shape) SQLQuack's tree, with a set operation's leading WITH moved from its first SELECT onto the set operation, where ClickHouse's parser does not put it no shape, so no column migration proof
Migration fingerprint the canonical tree, encoded as sorted JSON with a sqlquack-tree: prefix (SQLQuack has no SQL generator) no fingerprint, so no automatic rename
Raw diff query shape root kind, node kinds and write or lock clauses from SQLQuack's tree the query is refused as not one valid statement
Relation names tables, CTE names and temporary tables from every statement None, the existing "cannot parse" answer; empty SQL still gives no relations
Fixture star keys projected stars in each SELECT: a qualified star takes its relation, and a bare star takes every relation in the query every relation given
Snowflake metadata-only CREATE the same CREATE rules over SQLQuack's tree not metadata-only, so billed as warehouse work
Cursor type of an unknown warehouse type read as a cast's type by SQLQuack's parser (DuckDB grammar) unclassified, with the existing warning

On this branch only ClickHouse takes the native route. Snowflake's native path for the cost check is in place but inactive until Snowflake joins the route.

Differences from the Polyglot path

These come from a before/after run over the 716 compiled ClickHouse model bodies in SQLBuild's e2e projects and seed corpus, plus 137 warehouse type spellings. "Before" is this branch's base with every proof on Polyglot; "after" is this branch.

  • Migration query shapes: identical on all 716 (projection names, references, passthrough, positional use, input relations, local columns and clause references).
  • Relation names: identical on all 716.
  • Fixture star keys: identical on all 215 bodies with a star.
  • Cursor types: identical on all 137 spellings.
  • Raw diff query shape:
    • Root kind and write or lock clauses are identical on all 716.
    • The read-only verdict is identical: the node kinds differ only in function nodes (for example a generic function instead of upper), and no mutating kind appears or disappears.
    • Refusal messages for empty SQL differ in wording only (30 bodies): the clickhouse parser cannot read Empty query instead of Expected 1 statement, found 0.
  • Migration fingerprints: the same 30 bodies get none.
    • The native grouping refines Polyglot's: no two bodies Polyglot fingerprinted differently now match.
    • 3 Polyglot groups (30 models) split, because Polyglot renders VARCHAR(19) and VARCHAR(23) as the same ClickHouse type and the native tree keeps the length. So a native match is always a pair Polyglot also called equal.

Release note

ClickHouse model renames are matched on a new migration fingerprint. A ClickHouse model renamed in the same change as this upgrade is not matched automatically against fingerprints stored by an earlier version. It is planned as a new model, as when no rename is found, and later renames match as before.

Verification

  • cargo test -p sqlbuild-analysis: 310 passed, 0 failed. This includes new tests for each native proof and for each one's refusal on SQL with a gap (INTO OUTFILE).
  • The arm's type test compares the full NormalizedType, family included, against Polyglot's on 99 ClickHouse spellings.
  • cargo clippy -p sqlbuild-analysis -p sqlbuild-python --all-targets -- -D warnings: clean.
  • fensu check: 0 faults. Its Rust custom-rule host times out at 30 s in this environment, and does the same on an untouched checkout.
  • The before/after corpus run is described above.

Migration query shapes and fingerprints, raw diff query shapes, relation
names, fixture star keys and Snowflake's metadata-only CREATE now parse a
native dialect's SQL with SQLQuack, and keep Polyglot only on the legacy
route. SQL the native parser cannot read whole gets no proof, so the
planner takes its conservative path. Cursor types parse unknown warehouse
types with SQLQuack's parser.

NativeDialect gains parse, and read_whole becomes a provided method over
it, as the analysis route's follow-up defines them.

kvlonge commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #1363: the same commit on a branch name that passes the metadata check, with a shorter description.

@kvlonge kvlonge closed this Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant