Skip to content

feat: convert PIVOT without an alias column list - #105

Closed
kvlonge wants to merge 274 commits into
mainfrom
feat/pivot-unaliased
Closed

kvlonge wants to merge 274 commits into
mainfrom
feat/pivot-unaliased

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Why

PIVOT(...) without an alias column list is how dbt models usually write it, and it was refused. Recorded Snowflake receipts now show how Snowflake names those columns.

Changes

  • A PIVOT with no alias column list keeps the other columns' names and names each key column by its text in single quotes, with case and spaces as written, for example 'web' and 'Email Opt'. Quoted read-back such as "'web'" works.
  • A key holding a quote or backslash, whose column name no recording shows, is still refused, as is a partial alias list.
  • The PIVOT rule's catalogue conditions describe the new limits.
  • New recorded cases: cases/relational/pivot_unaliased.toml (4 cases), with Snowflake receipts.

Verification

  • The CLI test checks the converted column names; the conversion test covers the quote refusal and the partial list.
  • Local replay of every recorded case: 8,476 cases, 0 failed; all 4 new cases pass.
  • Static checks pass locally. Merge gate before landing.

Work in progress. Parses TruncateStmt and both CopyStmt forms (old
option words and the parenthesized generic list) into the shared
Truncate and CopyStmt nodes, with gram.y's STDIN/STDOUT-with-PROGRAM
and WHERE-with-COPY-TO errors; COPY FROM ... WHERE stays a gap.
Against the 18.6 receipts: 5139 parsed and 239 opaque agree, 0
disagreements; Postgres-rejected inputs accepted as opaque fall from
254 to 180.
The Postgres referee judges a case whose text splits into several
statements on its whole text and records parser_check = whole_text.
Format /2 roots accept it alongside do_block and sql_function; /1
roots are unchanged.
Cargo-generated Cargo.lock entry for sqlquack-syntax-postgres so
--locked builds pass with the workspace member line; no existing
dependency changes.
The referee now judges a text that splits into several statements on its whole text
(parser_check whole_text); all three are syntax errors, as expected, and two fresh-container runs
match. Nothing is pending: all 14,595 cases have receipts.
dialect.toml may set local_parser = "postgres": the isolated worker
then runs sqlquack-syntax-postgres (parse-worker --postgres). A gap,
Command or Raw node makes a parse opaque, so it is never verified; a
raw-parser error other than 42601 is opaque with no tree, since
PostgreSQL does not report it as a syntax error. The Snowflake worker
command and verdicts are unchanged.
A thin public wrapper over the crate's statement findings, beside
statement_lineage, so callers holding a tree from another parser can
get E201, E221 and E222 findings without reparsing. No behaviour
change.
Snapshot of the Postgres root judged by the Postgres parser (wip/pg-local-parser 7a9066b):
7,908 of 22,217 obligations satisfied, 0 forms verified until reviews exist, and the 5,552
Postgres-rejected cases the parser accepts as opaque recorded as known contradictions.
Work in progress. Parses MergeStmt into the shared Merge node: the
target with its alias, a USING table reference that may be a join,
WHEN [NOT] MATCHED [BY SOURCE|TARGET] clauses with AND conditions, the
UPDATE/DELETE/INSERT/DO NOTHING actions as Polyglot shapes them, and
RETURNING. INSERT is refused for matched rows and UPDATE/DELETE for
unmatched ones, as gram.y does. All 80 MERGE receipt cases agree.
Refactor only. Moves the grammar helpers into tree (cursor, node
builders, type names), values (expressions, primaries, calls) and
commands (queries, FROM, DML, statements), leaving room for the
SQL/JSON and SQL/XML grammar. Verdicts against the 18.6 receipts are
unchanged.
Work in progress. The SQL/JSON constructors, query functions and
aggregates, the SQL/XML calls, TREAT, XMLTABLE and JSON_TABLE (also
after LATERAL) are now read with gram.y's full grammar and its own
errors (22023 for an unknown JSON encoding, 0A000 for a non-constant
JSON_TABLE path, 42601 for repeated XMLTABLE column options), instead
of being skipped. They stay opaque gaps in the tree. Against the 18.6
receipts (wip/pg-catalogue 4ddc2fc), 154 Postgres-rejected JSON/XML
cases that were accepted as opaque are now rejected; 0 disagreements.
Work in progress. GROUP BY DISTINCT is held as the group's quantifier
(GroupBy.all = Some(false)); GROUP BY ALL stays unmarked, since the
shared tree's Some(true) means Snowflake's GROUP BY ALL. SET (a, b) =
(x, y) or ROW(x, y) with as many items is split into one assignment
per target, as PostgreSQL does; other sources stay a gap. WHERE
CURRENT OF, RETURNING WITH and locking on a set operation have no
field in the shared tree and stay gaps.
wip/pg-local-parser 28550cb: 8,119 of 22,217 obligations (211 gained, none lost against the
first baseline), 5,372 Postgres-rejected cases still accepted as opaque.
Tokens carry their range in characters with the line and column after
them, as the shared tree records spans. Names, columns, expressions,
targets, queries, CTEs, joins and FROM items fill their span fields;
a query's range covers its WITH clause and trailing options.
Engine::Postgres follows the PostgreSQL rules of the validator SQLBuild
uses: no predicate, comparison or string set-operation coercions, its
interval and date/time arithmetic overloads, strict NULLIF unification,
and no recursive-CTE or interval-literal checks. Where PostgreSQL 18.6
contradicts those rules, the checks follow 18.6: impossible casts and
duplicate CTE names are errors, stored tables offer system columns, and
output names are read only by GROUP BY items and bare ORDER BY keys.

Snowflake and DuckDB checks are unchanged.
A projection without an alias takes the name FigureColname gives it:
a column's or field's name, a function's name, a cast's operand name or
else its type name, CASE's ELSE name or else case, a scalar subquery's
first output, and ?column? for anything else. Snowflake and DuckDB
naming is unchanged.
Engine::Postgres now types string literals and || as text, and built-in
calls by resolving their overload against pg_proc and pg_type, read from
the pinned postgres:18.6 image by scripts/extract_postgres_catalog.py.
coalesce, greatest, least and CASE take their common type, and the SQL
value functions (current_date and so on) take gram.y's types. Snowflake
and DuckDB typing is unchanged.
The other columns keep their names and each key column is named by its
text in single quotes, case and spaces as written. A key holding a quote
or backslash, or a partial alias list, still refuses.
Snowflake correlates TABLE(FLATTEN(t.col)) f to the sources before it, as
dbt models write it; it now converts through the LATERAL FLATTEN rule,
with a positional or named input.
Engine::Postgres now types arithmetic, unary minus and NULLIF by picking
the operator PostgreSQL picks from pg_operator, with implicit casts from
pg_cast and preferred types from pg_type. CASE, COALESCE, GREATEST,
LEAST and set operations take select_common_type's type, keeping type
modifiers only when every value has them. Integer literals are integer,
bigint or numeric by size, and casts keep PostgreSQL's spelling (real,
text, char(n), numeric(p,s), timestamp(p), arrays).

scripts/record_postgres_types.py records the result type of 1,987
synthetic expressions through temporary views on the pinned 18.6 image,
and the semantics tests replay every typed one: none is typed wrongly
and 1,315 of 1,367 are typed.
Snowflake results for the six cases in cases/relational/pivot_unaliased.toml and cases/semi-structured/table_flatten.toml at a20177b.
PostgreSQL refuses a window function with no OVER clause (42809). The
placement rule already reported E232 for the window nodes every engine
parses, so Engine::Postgres now also reports it for the calls that rule
missed: a built-in window function (prokind w in the 18.6 pg_proc
extract) called by name, as the Postgres parser builds it, and
cume_dist, percent_rank and nth_value. A call under OVER or WITHIN
GROUP is not reported.
TransformKind now orders the steps from least to most significant:
constant, direct, cast, expression, window, aggregation. An output takes
the most significant step on its path. Constant applies only when the
path reads no column and passes through no aggregate or window, so
COUNT(*) stays aggregation and COUNT(*) OVER () is window.

- Window functions are window, not aggregation or expression; a group
  aggregate in their arguments makes them aggregation.
- A set operation, VALUES column or UNPIVOT value takes its most
  significant branch, and a constant branch is no step, so branches
  that each pass a column through are direct.
- A branch whose type the set operation coerces counts as a cast.

This changes the kinds of existing Snowflake and DuckDB outputs:
unions, INTERSECT, VALUES, UNPIVOT and recursive CTE pass-throughs that
were expression are now direct, and window functions that were
aggregation or expression are now window.
parse_type reads a type string as PostgreSQL's typeStringToTypeName does,
and format_type names a type as format_type does; 115 of 130 type strings
agree with 18.6's receipts, and the rest fail only at catalogue lookup
(unknown names, out-of-range modifiers) or carry an interval precision
the shared tree cannot hold. Built-in generic names, also pg_catalog
qualified and with modifiers, become their shared types.

__polyglot_typed(expr, 'type') and __polyglot_typed_like(expr, like)
become Annotated nodes, as sqlquack-syntax lowers them.

A qualified star's qualifier now records its source range.
…grammar for two publication negatives

set-role.none-reset-role and set-session-authorization.default-reset-
session-authorization came from a synopsis reader that joined a RESET
line onto the SET line before it; 19aa1f2 split each into the two
documented forms and kept the old id as absent_from_source. aliases.toml
now records each split, which the checker validates. Two create-
publication notes cited preprocess_pubobj_list for what are plain
grammar errors; o046 and o047 are retired for o049 and o050.
…d refused

ROLLBACK [ TRANSACTION ] TO SAVEPOINT and RELEASE SAVEPOINT name a
savepoint called savepoint, and DEALLOCATE PREPARE a statement called
prepare: the optional keyword is left out and the unreserved keyword is
the name. 18.6 parses all four (25P01, 26000 at run time). Run
pg-18.6-20261106T171708300305Z and its repeat agree.
Batch 28 re-approves create-publication after notes-8. The baseline is
re-recorded at wip/pg-integration c55b7bc: 1,384 of 1,555 forms,
19,346 of 22,726 obligations, 0 problems.
…count every implicit coercion as a cast"

This reverts commit 13d37b7.
…rser still keeps as one

CREATE TABLE has parsed to a full tree since 2c68336, so the case no longer
showed a gap. The dialect test's user-at-host fixture is now alice@host.local.
# Conflicts:
#	crates/sqlquack-syntax-coverage/src/cli/_helpers/results_command.rs
…structured columns a query reads

The built-in default now converts CURRENT_DATE() and CURRENT_TIMESTAMP() with a warning that
states the caveat: both engines take the value when the query runs, in the session's time zone.
Strict mode, --loud-divergences-only and a session outside UTC still refuse them.

A base table's VARIANT, ARRAY and OBJECT columns are guarded, relying on json_fidelity, only
where the statement can read them: it names the column, or reads columns without naming them
through a star other than COUNT(*), a positional column, PIVOT, UNPIVOT or a NATURAL join.
Reading another column of such a table no longer needs json_fidelity.
A comment straight after SELECT set post_select_comments, which no rule
read, so the whole statement refused (602 of the playground's 3,263
generated statements). It is now cosmetic like every other comment
field. A comment after an expression wrapped it in an annotated node,
so a rule needing a literal, such as ROUND's scale, saw a wrapper and
refused; a wrapper that holds only comments is now unwrapped. Both are
dropped from the output, as comments elsewhere are.
Unwrapping comment-only wrappers dropped the comment from an unaliased
literal's written text, so SELECT 'x' /* c */ took the name 'X', which no
receipt records. Main had the same gap. The select item's own wrapper now
stays, and the projection rule refuses it with SQ0009 unless it wraps a
column, star or alias, as the column-name rule's conditions state.
No behaviour change: the select-item wrapper rule from the previous commit
now lives in unwrap_field, and the inline comment and long doc comment go.

kvlonge commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Closing: SQLQuack lands through local-gate trains, not PR merges (D94), so open PRs only add clutter (D628/D630). The branch is kept. Any work here that isn't on main yet is tracked by the train owner and lands through a train.

@kvlonge kvlonge closed this Nov 8, 2026
@kvlonge
kvlonge deleted the feat/pivot-unaliased branch November 9, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant