Repository navigation
Postgres step 1: the PostgreSQL parser, catalogue and semantics profile (D289) - #131
Merged
Merged
Conversation
Only a dialect other than Snowflake may name a parser path that is not committed yet; it then has no parser tree and every review is stale.
coverage/syntax-postgres/dialect.toml is the per-root checker settings file agreed with the checker lane. focus/postgres.toml lists the 44 catalogued forms SQLBuild's Postgres test SQL uses (at sqlbuild 9082be13), in work order, each with the plan purposes A-H that need it; the header states how the SQL was collected and classified with the 18.6 parser.
scripts/syntax_obligations_pg runs the shared obligation builder on the Postgres root: per-family submission contexts, placeholder fills, class rules for typed-value placeholders and an upper-cased grammar read. coverage/syntax-postgres/limits.toml holds the readings, suppressions and hand-written lexical limits with reasons. Obligations cover all 8 families; cases cover the 44 focus forms (424 cases) in the referee's case shape.
417 of the 424 focus cases recorded against the pinned PostgreSQL 18.6 image by the Postgres referee: every accept and reject expectation agrees with the server (237 syntax errors, 180 parsed). The case build now reads the root's receipts for observed outcomes and later stages. The 7 unterminated-token negatives wait on a referee fix.
Constants, models and a domain exception move to their roles, dataclasses are frozen, parameters keyword-only, the audit's global lead table becomes a returned value, and entry modules keep one public function. Outputs are byte-identical: the catalogue, audit counts, obligations and cases do not change.
Receipt format /2 roots take exactly execute_in_rolled_back_transaction and execute_autocommit_in_throwaway_database for case captures and receipt capture methods; format /1 roots refuse them.
Cases now cover all of query and dml besides the focus forms (749 cases). Generic grammar rewrites (ONLY or *, LIMIT or FETCH, DISTINCT with a list, with_query as name AS (query), explicit second list items) and readings for INSERT and legacy COPY make the renders valid; suppressions record valid neighbours. 560 new receipts agree with the server; 12 cases are pending referee fixes (TABLE in the DO-block pass, analysis-time 42601 for SELECT INTO) and are listed in pending.toml.
…overage The Postgres referee now parses statements PL/pgSQL takes first as a SQL function body, so parser_check is do_block or sql_function; receipts from the retired live_only check are refused.
…arse pass The 560 query and dml receipts of run pg-18.6-20261106T010754435065Z included 103 decided by the live run (parser_check live_only), which the checker no longer takes. The same cases and the 12 pending TABLE and SELECT ... INTO cases are recorded again with the referee at c22b085 as run pg-18.6-20261106T011602128783Z: 572 receipts, 452 do_block and 120 sql_function, all agreeing with their cases; a second run gave the same 572. pending.toml is now empty.
Expressions joins query and dml in CASED_FAMILIES: 1,367 cases from 2,136 obligations. 1,202 new receipts (run pg-18.6-20261106T013025158231Z, referee c22b085) agree with their cases, and a second run gave the same receipts. - The grammar is tokenized with Postgres operators: a run of operator characters is one token, a placeholder never holds only operator characters, and \[ \] in a reading is a literal bracket. - Data types render inside CAST ( NULL AS ... ) by form prefix; operator operands named for a type take a value of that type. - Readings for array constructors, subscripts, slices, array types, numeric, interval and "any"; hand-written limits for the twelve remaining lexical forms. - 180 derived negatives that are valid PostgreSQL are withdrawn with their gram.y reasons; 120 are operators with the left operand left out, which gram.y takes as prefix operators. - The catalogue keeps the <-> operator in its two signatures, which tag removal had dropped. - One case is held in pending.toml: its message names the @-@ operator, which the checker's address rule refuses.
PostgreSQL messages echo operator names such as @-@, @>, <@ and @@, which the bare @ rule refused. Receipts of format /2 now refuse an @ only between word characters, as in an email address or a user@host identity; format /1 keeps the bare @ rule.
… yet) Work in progress, not a workspace member yet and never compiled. Holds the kwlist.h keyword table, the scan.l-following scanner with its lookahead filter, the parser cursor and name classes, shared-tree node builders and the Typename grammar. The expression, SELECT and DML grammar and the parse entry point are still to come.
Functions joins CASED_FAMILIES: 3,442 cases from 5,252 obligations. 3,406 new receipts (run pg-18.6-20261106T014128439859Z, referee c22b085) agree with their cases, and a second run gave the same receipts. - The catalogue cuts a signature at its return value, so a set-returning function's output columns are no longer read as call syntax, and pairs a parameter with a replaceable type as with a type tag (54 functions forms change, ids unchanged). - Function parameters written <name type> are expressions (gram.y func_arg_expr). - Readings for parameter defaults, the SQL/JSON constructors and XMLTABLE's column list; XMLTABLE and JSON_TABLE render in FROM. - 31 derived negatives that are valid PostgreSQL are withdrawn with their gram.y reasons. - Run pg-18.6-20261106T005058205176Z held 47 receipts of cases since changed; its 370 current cases are recorded again as run pg-18.6-20261106T015007294810Z (identical receipts) with the @-@ case the checker now accepts. pending.toml is empty.
…stgreSQL crate Work in progress. Adds the gram.y expression grammar by precedence climbing, queries with set operations, WITH and the ORDER BY/LIMIT/FETCH/locking checks of insertSelectOptions, the FROM clause with joins, LATERAL and function tables, the DML statements with ON CONFLICT and RETURNING, and the parse entry point. Statements and constructs the shared tree lacks become opaque Command statements, listed in the README. Adds the crate to the workspace members; Cargo.lock is left unchanged and the build regenerates it locally. fensu still reports faults in this crate.
Work in progress. Moves ParseError to parse/errors.rs, statement dispatch to _helpers/statements.rs and type names to typenames.rs, compacts the keyword table, and re-exports parse, ParseError, Gap and Parsed from the crate root. No grammar change: agreement with the 18.6 receipts is unchanged (5013 parsed, 291 opaque, 0 disagreements).
Work in progress. Groups long parameter lists (SelectOptions, TableAlias, JoinConstraint, LikeTest, tuple bounds), names the literals and thresholds that drive decisions, replaces .ok() probes with a checked digit fold (scan::main::digits), renames is_test and is_null after what they build, and keeps the verdict test branch-free. No grammar change: agreement with the 18.6 receipts is unchanged (5013 parsed, 291 opaque, 0 disagreements); fensu reports 0 faults.
…brace action choices The synopsis reader joined RESET ROLE and RESET SESSION AUTHORIZATION onto the line before, so two forms read SET ... NONE RESET ROLE. RESET now starts a statement on SET pages; the joined ids stay as absent_from_source and four forms replace them. ALTER FUNCTION's and similar action alternatives now carry their braces; no form id changes.
…admin form 14,595 cases across all eight families now; 14,592 have receipts from referee runs on the pinned 18.6 image. The four new families ran in fresh containers, each repeated in a second fresh container with identical receipts; runs.toml records the isolation and the start and end state. Three ddl negatives stay pending on a referee decision. Fills for transaction modes, lock modes, role options, large objects and default privileges; suppressions for derived negatives that are valid PostgreSQL, each with its gram.y reason; capture patterns follow the referee's widened autocommit list.
Work in progress. Parses TruncateStmt and both CopyStmt forms (old option words and the parenthesized generic list) into the shared Truncate and CopyStmt nodes, with gram.y's STDIN/STDOUT-with-PROGRAM and WHERE-with-COPY-TO errors; COPY FROM ... WHERE stays a gap. Against the 18.6 receipts: 5139 parsed and 239 opaque agree, 0 disagreements; Postgres-rejected inputs accepted as opaque fall from 254 to 180.
The Postgres referee judges a case whose text splits into several statements on its whole text and records parser_check = whole_text. Format /2 roots accept it alongside do_block and sql_function; /1 roots are unchanged.
Cargo-generated Cargo.lock entry for sqlquack-syntax-postgres so --locked builds pass with the workspace member line; no existing dependency changes.
The referee now judges a text that splits into several statements on its whole text (parser_check whole_text); all three are syntax errors, as expected, and two fresh-container runs match. Nothing is pending: all 14,595 cases have receipts.
dialect.toml may set local_parser = "postgres": the isolated worker then runs sqlquack-syntax-postgres (parse-worker --postgres). A gap, Command or Raw node makes a parse opaque, so it is never verified; a raw-parser error other than 42601 is opaque with no tree, since PostgreSQL does not report it as a syntax error. The Snowflake worker command and verdicts are unchanged.
A thin public wrapper over the crate's statement findings, beside statement_lineage, so callers holding a tree from another parser can get E201, E221 and E222 findings without reparsing. No behaviour change.
Snapshot of the Postgres root judged by the Postgres parser (wip/pg-local-parser 7a9066b): 7,908 of 22,217 obligations satisfied, 0 forms verified until reviews exist, and the 5,552 Postgres-rejected cases the parser accepts as opaque recorded as known contradictions.
Work in progress. Parses MergeStmt into the shared Merge node: the target with its alias, a USING table reference that may be a join, WHEN [NOT] MATCHED [BY SOURCE|TARGET] clauses with AND conditions, the UPDATE/DELETE/INSERT/DO NOTHING actions as Polyglot shapes them, and RETURNING. INSERT is refused for matched rows and UPDATE/DELETE for unmatched ones, as gram.y does. All 80 MERGE receipt cases agree.
The check that every byte is in a token, a space or a comment becomes a function over token ranges, so a test can give it ranges that skip a byte.
Batch 25 re-approves the 16 forms notes-6 moved. The baseline is re-recorded at wip/pg-integration da3f754: 1,384 of 1,555 forms, 19,330 of 22,706 obligations, 0 problems.
…ions The foreign partition form gets NOT NULL NO INHERIT, GENERATED VIRTUAL and without a kind, CHECK ENFORCED and NOT ENFORCED as column and table constraints, and NOT NULL col NO INHERIT, named and unnamed. CREATE PUBLICATION gets seven malformed-object negatives, each 42601 in 18.6. Run pg-18.6-20261106T160927505389Z and its repeat agree on all 16 cases.
…quack into wip/pg-integration
At wip/pg-integration 5348cab: 1,384 of 1,555 forms, 19,330 of 22,706 obligations, 822 contradicted, 0 problems; no approved fingerprint moved.
…grammar for two publication negatives set-role.none-reset-role and set-session-authorization.default-reset- session-authorization came from a synopsis reader that joined a RESET line onto the SET line before it; 19aa1f2 split each into the two documented forms and kept the old id as absent_from_source. aliases.toml now records each split, which the checker validates. Two create- publication notes cited preprocess_pubobj_list for what are plain grammar errors; o046 and o047 are retired for o049 and o050.
…d refused ROLLBACK [ TRANSACTION ] TO SAVEPOINT and RELEASE SAVEPOINT name a savepoint called savepoint, and DEALLOCATE PREPARE a statement called prepare: the optional keyword is left out and the unreserved keyword is the name. 18.6 parses all four (25P01, 26000 at run time). Run pg-18.6-20261106T171708300305Z and its repeat agree.
Batch 28 re-approves create-publication after notes-8. The baseline is re-recorded at wip/pg-integration c55b7bc: 1,384 of 1,555 forms, 19,346 of 22,726 obligations, 0 problems.
…ensu rules" This reverts commit 0f203a0.
…count every implicit coercion as a cast" This reverts commit 13d37b7.
This reverts commit 5ceec19.
…rser still keeps as one CREATE TABLE has parsed to a full tree since 2c68336, so the case no longer showed a gap. The dialect test's user-at-host fixture is now alice@host.local.
# Conflicts: # crates/sqlquack-syntax-coverage/src/cli/_helpers/results_command.rs
Contributor
Author
|
Landing gate (D289 step 1, D94 slot after #127) on the exact head
Postgres figure (parser tree d7df6560): 1,384 of 1,555 forms verified (89.0%), 19,346 of 22,726 obligations, 0 problems. Landing by fast-forward of main to |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Step 1 of putting PostgreSQL support on main (D289). This lands the PostgreSQL 18 parser crate, its syntax catalogue and verification data, and the Postgres profile in sqlquack-semantics. It lands as code and figures only. Nothing user-facing claims Postgres support.
Changes
crates/sqlquack-syntax-postgres: a scanner and parser that follow PostgreSQL 18.6's gram.y. Statements it doesn't model yet stay opaque gaps instead of being guessed.coverage/syntax-postgres: the Postgres catalogue, obligations, cases, receipts and reviews, checked bysyntax-coverage --root coverage/syntax-postgres.sqlquack-semantics: the Postgres engine profile, type data and output naming. Every new path applies only under the PostgreSQL profile.sqlquack-syntax-coverage: Postgres verdicts and checks. Snowflake-only commands, such asresults, refuse other roots.Only the coverage checker depends on the Postgres parser crate, so no product crate, Python module or wasm build can reach a Postgres tree (D118). sqlquack-syntax (the Snowflake parser) is unchanged at tree a66f33a4.
Figures
Postgres catalogue at parser tree d7df6560: 1,384 of 1,555 forms verified (89.0%), and 19,346 of 22,726 required obligations satisfied. The 2 absent forms are alias splits recorded in aliases.toml.
Verification
On the gated head, against main built the same way:
make check-ciandmake test-rustpass.staleis empty.translation-results --against mainshows no regression.Audit: approved in round 2 (record 7070d59).