Skip to content

feat(types): normalize adapter types without Polyglot - #1366

Closed
kvlonge wants to merge 3 commits into
feat/sqlquack-type-and-local-sqlfrom
feat/sqlquack-type-normalization
Closed

kvlonge wants to merge 3 commits into
feat/sqlquack-type-and-local-sqlfrom
feat/sqlquack-type-normalization

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #1364, whose type parsers this reuses.

Why

Adapter type normalization was the last type path that parsed with Polyglot, and its Python binding raised the wheel's PolyglotError. Contract and scenario checks compare types through it, so the replacement must keep the same aliases equal.

Changes

  • Types are read by SQLQuack's parsers where a dialect has one (Snowflake, DuckDB, SQLite, Postgres, BigQuery). ClickHouse, Databricks, SQL Server, and spellings the parsers reject, use a new Polyglot-free reader. Under ClickHouse it reads nested type arguments, enum members and named tuple fields.
  • A new speller writes each dialect's canonical spelling, which same_type compares. Only shipped dialects are known. motherduck reads as DuckDB, and other names raise Unknown dialect.
  • Types nested over 512 levels fall back to text; deep types run on their own stack.
  • normalize_type raises only ValueError. PolyglotError and the Polyglot type-argument helper are gone.
  • A test asserts same_type for every alias group in the Polyglot baseline: 276 groups, 800 spellings, 1,407 pairs.

These pairs no longer compare equal, as corrections:

  • Postgres FLOAT / REAL: bare FLOAT is DOUBLE PRECISION.
  • BigQuery TIMESTAMP / TIMESTAMP WITHOUT TIME ZONE: civil time is DATETIME.
  • BigQuery STRUCT(a TIMESTAMP) / STRUCT(a TIMESTAMP WITHOUT TIME ZONE).
  • MySQL's nested TIMESTAMP_LTZ pair is gone, because MySQL is not a shipped dialect.

The contracts golden drops two false Postgres K002s: int4 declared against inferred INT, and INT against int4.

Verification

  • cargo test -p sqlbuild-analysis --lib, clippy and fmt.
  • Python tests over adapters, contracts, semantic checks and scenario capture, on a fresh native build.
  • Polyglot baseline, 28,548 outcomes: 22,655 unchanged, and 159 valid and 1,109 invalid spellings normalize differently, listed in a comment. motherduck's 2,331 now equal DuckDB's, and mysql's 2,294 now raise.

Read each type with SQLQuack's parser for the dialect, or SQLBuild's own
type reader where a dialect has none, and spell it the way the
Polyglot-based normalization did so the same aliases stay equal.
normalize_type now raises only ValueError, for an unknown dialect.

kvlonge commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Moves against the Polyglot baseline, updated for 9c09e54 (28,548 outcomes: 22,655 unchanged; motherduck's 2,331 now equal DuckDB's; mysql's 2,294 now raise Unknown dialect; 159 valid and 1,109 invalid spellings normalize differently). The tables below were taken at 2d146f5; the mysql rows no longer apply.

Contract equality. Every pair of valid spellings the baseline normalized alike still compares equal (276 groups, 800 spellings, 1,407 pairs; asserted by given_spellings_of_one_type_when_comparing_then_every_pair_is_the_same_type), except these named corrections, which no longer compare equal:

Dialect Pair Why
postgres FLOAT / REAL Postgres FLOAT without a precision is DOUBLE PRECISION.
bigquery TIMESTAMP / TIMESTAMP WITHOUT TIME ZONE (and in a STRUCT) BigQuery's TIMESTAMP is absolute time; civil time is DATETIME.

New equalities on valid spellings, all aliases the baseline missed: Postgres int4≡INT, int8≡BIGINT, float8≡DOUBLE PRECISION, DEC≡DECIMAL, CHAR≡CHAR(1), FLOAT≡DOUBLE PRECISION; BigQuery TINYINT≡INT64, TIMESTAMP≡TIMESTAMP WITH TIME ZONE, DATETIME≡TIMESTAMP WITHOUT TIME ZONE, nested BIGINT≡INT64; precision-free nested TIMESTAMP_NTZ(n) as at top level. In the contracts golden this drops two false Postgres K002 errors (int4 against INT).

Dialects. Only shipped dialects are known: postgres, sqlite, snowflake, clickhouse, databricks, bigquery, tsql, duckdb, and motherduck, which reads and spells types as DuckDB, plus generic. Every other name, mysql included, raises Unknown dialect.

Valid spellings whose normalized text moved (canonical dialect names only):

Dialect Before After Spellings
bigquery ARRAY (other) ARRAY (other) BIGINT[]
bigquery ARRAY (other) ARRAY (other) ARRAY< TIMESTAMP(10) >
bigquery ARRAY<TIMESTAMPNTZ(0)> (other) ARRAY (other) ARRAY< TIMESTAMP_NTZ(0)>, TIMESTAMPNTZ(0) []
bigquery BLOB(10) (other) BLOB (other) BLOB(10)
bigquery DATETIME (timestamp) DATETIME (datetime) TIMESTAMP WITHOUT TIME ZONE
bigquery DATETIME (timestamp) TIMESTAMP (timestamp) TIMESTAMP, TIMESTAMP(0), TIMESTAMP(10), timestamp …
bigquery HUGEINT(10) (other) HUGEINT (other) HUGEINT(10)
bigquery MAP(TIME(10),(38,0)) (other) MAP (other) MAP(TIME(10), ( 38 , 0 ))
bigquery MAP<TIMESTAMPNTZ(10),TIMESTAMPNTZ(0)> (other) MAP<TIMESTAMPNTZ,TIMESTAMPNTZ> (other) MAP(TIMESTAMP_NTZ(10) , TIMESTAMP_NTZ(0) )
bigquery STRUCT<ABIGINT(10)> (other) STRUCT (other) STRUCT(a BIGINT(10) )
bigquery STRUCT (other) STRUCT (other) STRUCT(a BIGINT )
bigquery STRUCT (other) STRUCT (other) STRUCT(a TIMESTAMP(10) )
bigquery STRUCT<ATIMESTAMPNTZ(10)> (other) STRUCT (other) STRUCT(a timestampntz(10) )
bigquery TINYINT (integer) INT64 (integer) TINYINT
bigquery UUID(10) (other) UUID (other) UUID(10)
databricks ARRAY (other) ARRAY<FLOAT64(10,2)> (other) ARRAY< FLOAT64(10,2) >, FLOAT64(10,2) []
databricks ARRAY<TIMESTAMPNTZ(0)> (other) ARRAY (other) ARRAY< TIMESTAMP_NTZ(0)>, TIMESTAMPNTZ(0) []
databricks MAP<TIMESTAMPNTZ(10),TIMESTAMPNTZ(0)> (other) MAP<TIMESTAMPNTZ,TIMESTAMPNTZ> (other) MAP(TIMESTAMP_NTZ(10) , TIMESTAMP_NTZ(0) )
databricks STRUCT<A:BIGINT,B:DOUBLE> (other) STRUCT<A:INT64(10),B:FLOAT64(10,2)> (other) STRUCT<a INT64(10), b FLOAT64(10, 2) >
databricks STRUCT<A:TIMESTAMPNTZ(10)> (other) STRUCT<A:TIMESTAMPNTZ> (other) STRUCT(a timestampntz(10) )
duckdb DOUBLE[] (other) FLOAT64(10,2)[] (other) ARRAY< FLOAT64(10,2) >, FLOAT64(10,2) []
duckdb GEOGRAPHY(10) (other) GEOGRAPHY (other) GEOGRAPHY(10)
duckdb MAP(TIMESTAMPNTZ(10),TIMESTAMPNTZ(0)) (other) MAP(TIMESTAMPNTZ,TIMESTAMPNTZ) (other) MAP(TIMESTAMP_NTZ(10) , TIMESTAMP_NTZ(0) )
duckdb STRUCT(ABIGINT,BDOUBLE) (other) STRUCT(AINT64(10),BFLOAT64(10,2)) (other) STRUCT<a INT64(10), b FLOAT64(10, 2) >
duckdb STRUCT(AGEOGRAPHY(10)) (other) STRUCT(AGEOGRAPHY) (other) STRUCT(a geography(10) )
duckdb STRUCT(ATIMESTAMPNTZ(10)) (other) STRUCT(ATIMESTAMPNTZ) (other) STRUCT(a timestampntz(10) )
duckdb TIMESTAMPNTZ(0)[] (other) TIMESTAMPNTZ[] (other) ARRAY< TIMESTAMP_NTZ(0)>, TIMESTAMPNTZ(0) []
generic ARRAY (other) ARRAY<FLOAT64(10,2)> (other) ARRAY< FLOAT64(10,2) >, FLOAT64(10,2) []
generic ARRAY<TIMESTAMPNTZ(0)> (other) ARRAY (other) ARRAY< TIMESTAMP_NTZ(0)>, TIMESTAMPNTZ(0) []
generic MAP<TIMESTAMPNTZ(10),TIMESTAMPNTZ(0)> (other) MAP<TIMESTAMPNTZ,TIMESTAMPNTZ> (other) MAP(TIMESTAMP_NTZ(10) , TIMESTAMP_NTZ(0) )
generic STRUCT(ATIMESTAMPNTZ(10)) (other) STRUCT(ATIMESTAMPNTZ) (other) STRUCT(a timestampntz(10) )
generic STRUCT<ABIGINT,BDOUBLE> (other) STRUCT<AINT64(10),BFLOAT64(10,2)> (other) STRUCT<a INT64(10), b FLOAT64(10, 2) >
mysql "MIXEDCASE" (other) MIXEDCASE (other) "Mixed Case"
mysql "MIXEDCASE"(10) (other) MIXEDCASE(10) (other) "Mixed Case"(10)
mysql "MIXEDCASE"(10)[] (other) JSON (other) "mixed case"(10) []
mysql "MIXEDCASE"(10,2) (other) MIXEDCASE(10,2) (other) "Mixed Case"(10, 2), "Mixed Case"(10,2)
mysql "MIXEDCASE"(38,0) (other) MIXEDCASE(38,0) (other) "Mixed Case"( 38 , 0 )
mysql "MIXEDCASE"(MAX) (other) MIXEDCASE(MAX) (other) "Mixed Case"(MAX)
mysql "MIXEDCASE"[] (other) JSON (other) "Mixed Case" []
mysql "QUOTED" (other) QUOTED (other) "quoted"
mysql "QUOTED"(10) (other) QUOTED(10) (other) "quoted"(10)
mysql "QUOTED"(10,2) (other) QUOTED(10,2) (other) "quoted"(10, 2), "quoted"(10,2)
mysql "QUOTED"(38,0) (other) QUOTED(38,0) (other) "quoted"( 38 , 0 )
mysql "QUOTED"(MAX) (other) QUOTED(MAX) (other) "quoted"(MAX), "quoted"(max)
mysql GEOMETRYSRID10 (other) GEOGRAPHY(10) (other) GEOGRAPHY(10)
mysql GEOMETRYSRID4326 (other) GEOGRAPHY (other) GEOGRAPHY
mysql MAP<TIMESTAMPNTZ(10),TIMESTAMPNTZ(0)> (other) MAP<TIMESTAMPNTZ,TIMESTAMPNTZ> (other) MAP(TIMESTAMP_NTZ(10) , TIMESTAMP_NTZ(0) )
mysql STRUCT(ATIMESTAMP) (other) STRUCT(ATIMESTAMPLTZ) (other) STRUCT(a timestamp_ltz)
mysql STRUCT(ATIMESTAMPNTZ(10)) (other) STRUCT(ATIMESTAMPNTZ) (other) STRUCT(a timestampntz(10) )
mysql STRUCT<ABIGINT,BDOUBLE> (other) STRUCT<AINT64(10),BFLOAT64(10,2)> (other) STRUCT<a INT64(10), b FLOAT64(10, 2) >
postgres CHAR (string) CHAR(1) (string) CHAR, CHARACTER, char
postgres DEC (other) DECIMAL (decimal) DEC, dec
postgres DEC(10) (other) DECIMAL(10) (decimal) DEC(10)
postgres DEC(10,2) (other) DECIMAL(10,2) (decimal) DEC(10, 2), DEC(10,2)
postgres DEC(38,0) (other) DECIMAL(38,0) (decimal) DEC( 38 , 0 )
postgres DOUBLE[] (other) FLOAT64(10,2)[] (other) ARRAY< FLOAT64(10,2) >, FLOAT64(10,2) []
postgres FLOAT8 (other) DOUBLEPRECISION (float) float8
postgres INT4 (other) INT (integer) int4
postgres INT8 (other) BIGINT (integer) int8
postgres MAP<TIMESTAMPNTZ(10),TIMESTAMPNTZ(0)> (other) MAP<TIMESTAMPNTZ,TIMESTAMPNTZ> (other) MAP(TIMESTAMP_NTZ(10) , TIMESTAMP_NTZ(0) )
postgres REAL (float) DOUBLEPRECISION (float) FLOAT
postgres REAL(10) (float) REAL (float) FLOAT(10), REAL(10)
postgres STRUCT(ATIMESTAMPNTZ(10)) (other) STRUCT(ATIMESTAMPNTZ) (other) STRUCT(a timestampntz(10) )
postgres STRUCT<ABIGINT,BDOUBLE> (other) STRUCT<AINT64(10),BFLOAT64(10,2)> (other) STRUCT<a INT64(10), b FLOAT64(10, 2) >
postgres TIMESTAMPNTZ(0)[] (other) TIMESTAMPNTZ[] (other) ARRAY< TIMESTAMP_NTZ(0)>, TIMESTAMPNTZ(0) []
snowflake ARRAY(TIMESTAMPNTZ(0)) (other) ARRAY(TIMESTAMPNTZ) (other) ARRAY< TIMESTAMP_NTZ(0)>, TIMESTAMPNTZ(0) []
snowflake MAP(TIMESTAMPNTZ(10),TIMESTAMPNTZ(0)) (other) MAP(TIMESTAMPNTZ,TIMESTAMPNTZ) (other) MAP(TIMESTAMP_NTZ(10) , TIMESTAMP_NTZ(0) )
snowflake OBJECT(ATIMESTAMPNTZ(10)) (other) OBJECT(ATIMESTAMPNTZ) (other) STRUCT(a timestampntz(10) )
sqlite GEOGRAPHY(10) (other) GEOGRAPHY (other) GEOGRAPHY(10)
tsql MAP<TIMESTAMPNTZ(10),TIMESTAMPNTZ(0)> (other) MAP<TIMESTAMPNTZ,TIMESTAMPNTZ> (other) MAP(TIMESTAMP_NTZ(10) , TIMESTAMP_NTZ(0) )
tsql STRUCT(ATIMESTAMPNTZ(10)) (other) STRUCT(ATIMESTAMPNTZ) (other) STRUCT(a timestampntz(10) )
tsql STRUCT(AVARIANT(10)) (other) STRUCT(ASQL_VARIANT) (other) STRUCT(a VARIANT(10) )
tsql STRUCT<ABIGINT,BDOUBLE> (other) STRUCT<AINT64(10),BFLOAT64(10,2)> (other) STRUCT<a INT64(10), b FLOAT64(10, 2) >

MotherDuck now reads and spells types as DuckDB. MySQL and the other
SQL analysis dialects no adapter ships raise Unknown dialect instead of
reading as generic SQL. Under ClickHouse the type reader keeps nested
type arguments, enum members with values, and named tuple fields.

kvlonge commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Closing: every commit on this branch is already contained in #1374 (the combined parser train), which will land this change. The branch is kept; reopen if #1374 is dropped.

@kvlonge kvlonge closed this Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant