From 4c656ace6fe88b6105c18d54a9f0e8946ac38470 Mon Sep 17 00:00:00 2001 From: Kevin Longe <273747162+kvlonge@users.noreply.github.com> Date: Fri, 6 Nov 2026 00:03:23 +0000 Subject: [PATCH 001/156] WIP: PostgreSQL 18.6 syntax catalogue Adds coverage/syntax-postgres, built from the postgresql-18.6 release SGML by scripts/syntax_catalogue_pg, which reuses the Snowflake catalogue's grammar normaliser and id keeping by import: 1,546 forms across eight families, with a reviewed disposition for every synopsis block outside a command synopsis. Not yet done: the omission audit against gram.y and kwlist.h, and the fensu faults in the new scripts. --- coverage/syntax-postgres/README.md | 71 + coverage/syntax-postgres/catalogue/ddl.toml | 5047 +++++++++ coverage/syntax-postgres/catalogue/dml.toml | 203 + .../catalogue/expressions.toml | 4242 ++++++++ .../syntax-postgres/catalogue/functions.toml | 9658 +++++++++++++++++ coverage/syntax-postgres/catalogue/query.toml | 163 + .../catalogue/security_admin.toml | 1041 ++ .../catalogue/transaction_session.toml | 416 + .../syntax-postgres/catalogue/utility.toml | 415 + coverage/syntax-postgres/hand.toml | 1869 ++++ coverage/syntax-postgres/indexes.toml | 31 + coverage/syntax-postgres/sources.toml | 4244 ++++++++ scripts/build_syntax_catalogue_pg.py | 14 + scripts/fetch_syntax_docs_pg.py | 14 + scripts/syntax_catalogue_pg/__init__.py | 0 .../syntax_catalogue_pg/_helpers/__init__.py | 0 .../syntax_catalogue_pg/_helpers/catalogue.py | 387 + scripts/syntax_catalogue_pg/_helpers/fetch.py | 62 + .../syntax_catalogue_pg/_helpers/groups.py | 264 + scripts/syntax_catalogue_pg/_helpers/hand.py | 150 + scripts/syntax_catalogue_pg/_helpers/pages.py | 183 + scripts/syntax_catalogue_pg/_helpers/sgml.py | 103 + .../syntax_catalogue_pg/_helpers/synopsis.py | 127 + scripts/syntax_catalogue_pg/constants.py | 217 + scripts/syntax_catalogue_pg/main/__init__.py | 0 scripts/syntax_catalogue_pg/main/build.py | 80 + scripts/syntax_catalogue_pg/main/fetch.py | 17 + scripts/syntax_catalogue_pg/models.py | 101 + scripts/syntax_catalogue_pg/types.py | 5 + 29 files changed, 29124 insertions(+) create mode 100644 coverage/syntax-postgres/README.md create mode 100644 coverage/syntax-postgres/catalogue/ddl.toml create mode 100644 coverage/syntax-postgres/catalogue/dml.toml create mode 100644 coverage/syntax-postgres/catalogue/expressions.toml create mode 100644 coverage/syntax-postgres/catalogue/functions.toml create mode 100644 coverage/syntax-postgres/catalogue/query.toml create mode 100644 coverage/syntax-postgres/catalogue/security_admin.toml create mode 100644 coverage/syntax-postgres/catalogue/transaction_session.toml create mode 100644 coverage/syntax-postgres/catalogue/utility.toml create mode 100644 coverage/syntax-postgres/hand.toml create mode 100644 coverage/syntax-postgres/indexes.toml create mode 100644 coverage/syntax-postgres/sources.toml create mode 100644 scripts/build_syntax_catalogue_pg.py create mode 100644 scripts/fetch_syntax_docs_pg.py create mode 100644 scripts/syntax_catalogue_pg/__init__.py create mode 100644 scripts/syntax_catalogue_pg/_helpers/__init__.py create mode 100644 scripts/syntax_catalogue_pg/_helpers/catalogue.py create mode 100644 scripts/syntax_catalogue_pg/_helpers/fetch.py create mode 100644 scripts/syntax_catalogue_pg/_helpers/groups.py create mode 100644 scripts/syntax_catalogue_pg/_helpers/hand.py create mode 100644 scripts/syntax_catalogue_pg/_helpers/pages.py create mode 100644 scripts/syntax_catalogue_pg/_helpers/sgml.py create mode 100644 scripts/syntax_catalogue_pg/_helpers/synopsis.py create mode 100644 scripts/syntax_catalogue_pg/constants.py create mode 100644 scripts/syntax_catalogue_pg/main/__init__.py create mode 100644 scripts/syntax_catalogue_pg/main/build.py create mode 100644 scripts/syntax_catalogue_pg/main/fetch.py create mode 100644 scripts/syntax_catalogue_pg/models.py create mode 100644 scripts/syntax_catalogue_pg/types.py diff --git a/coverage/syntax-postgres/README.md b/coverage/syntax-postgres/README.md new file mode 100644 index 000000000..13cec69a9 --- /dev/null +++ b/coverage/syntax-postgres/README.md @@ -0,0 +1,71 @@ +# PostgreSQL syntax catalogue + +The denominator of SQLQuack's PostgreSQL syntax coverage: every documented PostgreSQL 18.6 syntax +form, one record per form, with a stable id `pg..[.]`. The layout and the +record formats are those of `coverage/syntax` (see `docs/syntax-coverage.md`); this file records +what differs. + +## Source + +The pinned source is the release tarball `postgresql-18.6.tar.bz2` (sha256 in `indexes.toml`), +not the rendered website. Its DocBook SGML is what postgresql.org/docs/18 is built from, so a +source id `doc.` names the page `https://www.postgresql.org/docs/18/.html`. +`indexes.toml` also binds the files the page list is read from (`reference.sgml`, +`ref/allfiles.sgml`, `filelist.sgml`, `postgres.sgml`) and the grammar the omission audit reads +(`src/backend/parser/gram.y`, `src/include/parser/kwlist.h`). + +A page is either + +- a SQL command reference page (a `refentry` of the SQL Commands part, 183 pages), or +- a `sect1` (or a chapter's introduction) of the SQL-language chapters: syntax, ddl, dml, + queries, datatype, json, array, rowtypes, rangetypes, func, typeconv, indices, textsearch, mvcc, + perform and parallel. + +`page_sha256` is the sha256 of that SGML text; `syntax_sha256` of its synopsis blocks and +signatures. + +## Regenerating + +```sh +python -m scripts.fetch_syntax_docs_pg CACHE # private cache, never committed +python -m scripts.build_syntax_catalogue_pg --cache CACHE --out coverage/syntax-postgres +``` + +The build writes nothing while any input is unresolved: a synopsis block with no disposition, a +disposition naming no block, a restatement of an uncatalogued form, or a hand form whose page +changed. PostgreSQL 19 is an in-place rebuild against its tarball; ids follow the same keeping +rules as the Snowflake catalogue (`scripts/syntax_catalogue/_helpers/forms.py`, reused by import). + +## Form rules + +1. **Command pages.** The `` synopsis is split into statements at each unindented + line that starts with the command's first word (SELECT also `WITH` and `TABLE`). Each + statement is a form. `where X is:` (and `and X is`, `X in ... is`) starts a + sub-grammar definition, `[[definition]]` `doc.::`; `can be one of` / `is one of` + lists its alternatives one per line. +2. **Actions.** A statement whose grammar takes ` [, ... ]`, where the page lists + `action` as alternatives (ALTER TABLE and six other ALTER pages), becomes one form per action, with the + action substituted into the statement. +3. **Other synopsis blocks** (on command pages outside the Synopsis, and in the chapters) each + have a reviewed disposition in `hand.toml` `[[block]]`, keyed by page and block sha256: + `form` (each production a form; on a command page, older syntax still accepted, named + `.