Skip to content

fix: align DuckDB and PostgreSQL type inference with engine results - #27

Merged
kvlonge merged 2 commits into
mainfrom
fix/duckdb-postgres-type-truth
Sep 26, 2026
Merged

kvlonge merged 2 commits into
mainfrom
fix/duckdb-postgres-type-truth

Conversation

@kvlonge

@kvlonge kvlonge commented Sep 26, 2026

Copy link
Copy Markdown

Why

DuckDB and PostgreSQL result types were never verified against the real engines, and many inferred types were wrong. For example, DuckDB / returns DOUBLE for every integer and decimal pair, but was inferred as an integer type. Downstream tools that compare inferred types with warehouse columns reported false schema changes as a result. Some DuckDB keyword functions (CURRENT_TIMESTAMP and others) and bare USER were also reported as unknown.

Changes

  • Engine ground-truth matrices, from a generator script, stored as test fixtures: DuckDB 1.5.2 (2,769 cases) and PostgreSQL 17 (1,282 cases).
  • Per-engine DuckDB/PostgreSQL type rules for arithmetic, literals, aggregates, CASE/COALESCE/NULLIF unification, date/time arithmetic, casts and common scalar functions. Unmodelled functions return no type instead of the first argument's type. Other dialects are unchanged.
  • DuckDB // parses as integer division. Bare USER parses as the niladic function where the dialect defines it. DuckDB keyword time functions are registered.
  • Validation accepts engine-verified DATE/TIME/TIMESTAMP arithmetic.
  • Version 0.13.4.

Verification

  • A replay test over both matrices: 0 mismatches. Before the fix, 1,000 DuckDB and 524 PostgreSQL cases were wrong or unknown.
  • Local runs of make fmt-check, lint-rust (Rust 1.98), check-consistency, extract-all-fixtures, all four test-rust-ci-* targets and test-python, all green.

Record DuckDB 1.5.2 and PostgreSQL 17 result types for a synthetic
expression matrix and replay them in tests. Add per-engine rules for
arithmetic (DuckDB `/` is DOUBLE, `//` parses as integer division),
literals, aggregates, CASE/COALESCE/NULLIF unification, date/time
arithmetic, casts and common scalar functions, returning no type where the
engine result is not modelled. Parse bare USER as the niladic function in
dialects that define it and register DuckDB's keyword time functions.
@kvlonge
kvlonge merged commit f0c8004 into main Sep 26, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant