Skip to content

Fix parser gaps: empty arguments, type parameters, reserved names, anonymous procedures, tree findings - #38

Merged
kvlonge merged 16 commits into
mainfrom
fix/parser-gaps-b
Nov 4, 2026
Merged

kvlonge merged 16 commits into
mainfrom
fix/parser-gaps-b

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Parser fixes from the ranked parser-gap list (#1, #3, #4, #12) and the auditor's D118 parser-tree findings.

Syntax (Snowflake agrees by receipt):

Tree (auditor D118 withholds):

  • An unquoted date part in DATEDIFF/TIMEDIFF/TIMESTAMPDIFF/EXTRACT(part, x)/TRUNC(x, part) is a date part, not a column.
  • ORDER BY ALL is OrderBy.all with its direction.
  • Unary + is kept as a Pos node.
  • Parenless CURRENT_ORGANIZATION_USER is a no_parens call.
  • Lineage checks treat Pos like Neg and read OrderBy.all. No recorded case uses unary + or ORDER BY ALL, and the converter refused both shapes before.

Coverage diff (syntax-coverage family <f> --json for each family, base parser 4391b71 vs this head; catalogue 7522b4e, evidence b326079, reviews wip/syntax-omission-audit e24f3c7):
forms completed +84, -0; obligations satisfied +456, -0.
ddl +16, dml +1, expressions +4, functions +45, query +1, scripting +15, transaction_session +2; utility obligations +229.

Forms gained
  • sf.ddl.drop-function
  • sf.ddl.drop-index
  • sf.ddl.drop-materialized-view
  • sf.ddl.drop-procedure
  • sf.ddl.drop-sequence
  • sf.ddl.drop-table
  • sf.ddl.drop-type
  • sf.ddl.drop-view
  • sf.ddl.undrop-database
  • sf.ddl.undrop-notebook
  • sf.ddl.undrop-schema
  • sf.ddl.undrop-snapshot
  • sf.ddl.undrop-streamlit
  • sf.ddl.undrop-table
  • sf.ddl.undrop-tag
  • sf.ddl.undrop-type
  • sf.dml.truncate-table.truncate
  • sf.expressions.data-types-datetime.time
  • sf.expressions.data-types-numeric.number
  • sf.expressions.data-types-text.varbinary
  • sf.expressions.data-types-user-defined
  • sf.functions.ai-complete-single-file
  • sf.functions.ai-complete-single-string
  • sf.functions.ai-count-tokens
  • sf.functions.ai-similarity
  • sf.functions.ai-translate
  • sf.functions.array-insert
  • sf.functions.decode
  • sf.functions.decrypt
  • sf.functions.decrypt-raw
  • sf.functions.encrypt
  • sf.functions.encrypt-raw
  • sf.functions.execute-ai-evaluation
  • sf.functions.h3-latlng-to-cell
  • sf.functions.h3-latlng-to-cell-string
  • sf.functions.haversine
  • sf.functions.insert
  • sf.functions.is-application-role-activated
  • sf.functions.is-group-activated
  • sf.functions.model-monitor-drift-metric.model-monitor-drift-metric
  • sf.functions.model-monitor-drift-metric.model-monitor-drift-metric-service-base-service
  • sf.functions.model-monitor-performance-metric.model-monitor-performance-metric-service
  • sf.functions.model-monitor-stat-metric.model-monitor-stat-metric
  • sf.functions.model-monitor-stat-metric.model-monitor-stat-metric-service
  • sf.functions.reduce
  • sf.functions.st-dwithin
  • sf.functions.st-transform
  • sf.functions.sys-context
  • sf.functions.sys-context-snowflake-application
  • sf.functions.sys-context-snowflake-organization
  • sf.functions.sys-context-snowflake-organization-sharing
  • sf.functions.sys-context-snowflake-session
  • sf.functions.system-get-default-columns-override-for-system-object
  • sf.functions.system-get-service-logs
  • sf.functions.system-get-tag
  • sf.functions.system-list-iceberg-tables-from-catalog
  • sf.functions.system-list-namespaces-from-catalog
  • sf.functions.system-tag-value-contains
  • sf.functions.to-decimal.to-decimal
  • sf.functions.to-decimal.to-number
  • sf.functions.to-decimal.to-numeric
  • sf.functions.try-decrypt
  • sf.functions.try-decrypt-raw
  • sf.functions.try-to-decimal.try-to-decimal
  • sf.functions.try-to-decimal.try-to-number
  • sf.functions.try-to-decimal.try-to-numeric
  • sf.query.with.select
  • sf.scripting.await
  • sf.scripting.case.end
  • sf.scripting.case.end.syntax
  • sf.scripting.declare.cursor-declaration
  • sf.scripting.declare.resultset
  • sf.scripting.for.do-end-for
  • sf.scripting.for.to-do-loop-end
  • sf.scripting.if
  • sf.scripting.let.let-cursor-for
  • sf.scripting.let.let-cursor-for.cursor-assignment-syntax
  • sf.scripting.let.let-default
  • sf.scripting.let.let-resultset-default
  • sf.scripting.loop
  • sf.scripting.repeat
  • sf.scripting.while
  • sf.transaction_session.use-database
  • sf.transaction_session.use-schema

Batched with #39 (D122)

This branch now also carries #39 (feat/unobservable-cause, head 75a66c7): cause = "not_submitted", readings in definition hashes, and receipt account_state. See #39 for details. Both are gated once, at this head; merging this PR lands #39's commits too.

Polyglot exact check

The parser fixes changed polyglot comparisons, which are now explained in corpora/polyglot-check/expected_differences.toml:

  • a new [[kept_node]] category, with one entry, unary_plus: the trees must be equal once SQLQuack's pos wrappers are removed;
  • Snowflake rejections: a CTE without AS, a reserved CTE or TRUNCATE name, an empty named argument, and reserved NOT as a typed-expression type (DuckDB keeps the original error).

Each of these is pinned by a parser test.

Coverage against the D122a floor

On main be4dfac: +84 forms, 0 lost, against a floor of 67 (2% of 3,348); +456 obligations, 0 lost.

Snowflake rejects calls such as TO_DECIMAL(1, ) and F(a, , b) with a syntax
error; the parser skipped the empty argument and accepted them.
Snowflake rejects TIME(), VARBINARY(), NUMBER(), NUMBER(9,) and PERIOD()
with a syntax error. CHAR(), VARCHAR() and STRING() stay accepted, as does
RETURNS TABLE ().
Snowflake rejects DROP TABLE SELECT, DROP INDEX v1.SELECT, UNDROP ... RENAME
TO SELECT, TRUNCATE SELECT, USE SELECT, DECLARE SELECT RESULTSET and
CAST(x AS SELECT); quoted names such as "SELECT" stay accepted. The check
applies to the names these statements parse, not to every table reference,
so FROM TABLE(...) and RETURNS TABLE (...) are unaffected.

Adds parser tests for the rejected statements and their accepted neighbours,
including the empty-argument and type-parameter fixes.
WITH <name> AS PROCEDURE (...) RETURNS ... LANGUAGE ... [options] [AS body]
followed by CALL <name>(...) [INTO :var] now parses to a structured Call
that keeps its WITH clause, instead of failing. The procedure is a
CreateProcedure CTE; RETURNS TABLE columns, handler options and the body
are kept, and a LANGUAGE SQL body is parsed as Snowflake Scripting so a
broken body fails. Snowflake CTEs also need AS and an unreserved name.

All 280 recorded WITH ... AS PROCEDURE cases now agree with Snowflake's
phase.
…tree

A faithful Snowflake parse now records what was written:
- an unquoted date part in DATEDIFF, TIMEDIFF, TIMESTAMPDIFF, EXTRACT(part, x)
  and TRUNC(x, part) is a date part, not a column; TRUNC(x, c1) stays a column;
- ORDER BY ALL is an OrderBy.all node with its direction, not a sort key;
- unary + is a Pos node instead of being dropped;
- CURRENT_ORGANIZATION_USER without parentheses is a no_parens call.

Lineage checks and typing treat Pos like Neg and read OrderBy.all. The WITH
clause's trailing statement moves to its own module to keep entry.rs within
its size limits.
An [[unobservable]] entry may set cause = "not_submitted" for cases that were
never sent to Snowflake, such as side-effecting calls a capture guard holds.
It has no runs or probes, and its obligation reports
unverifiable_not_submitted instead of unverifiable_internal_error, kept in the
denominator and never satisfied. An absent cause keeps today's 603 meaning;
any other value is refused.
…nt state

A form's definition hash now covers its [[reading]] from limits.toml, so
editing a reading stales that form's per-form review and no other. Forms
without a reading keep their hash. limits.toml is required; a duplicate
reading or one naming an uncatalogued form is refused.

Receipts may carry an optional account_state (D120): account parameters by
name, each with a value (or null) and a basis. It is validated, shown on
each case result, and changes no verdict.
A reading's grammar_sha256 may be a prose heading, as its form's is, and a
lexical form's reading may have no grammar lines when its reason explains
it. 54 readings in the catalogue have one or both.
Unary plus is now a kept_node entry: the trees must match once SQLQuack's
pos wrappers are removed. Snowflake-only rejections cover a CTE without AS,
a reserved CTE or TRUNCATE name, an empty named argument, and the reserved
NOT in a typed-expression type, each pinned by a parser test.
# Conflicts:
#	crates/sqlquack-syntax/tests/parser/test_fixes.rs

kvlonge commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Merged as a2c64d6 (parents 4a8656b, 736cf93) after a local gate (D94): make check-merge RECORDER=0 on head 736cf93 over main 4a8656b, exit 0. Merge tree fa0cb6e34b68dd70379645fc83222ad334c50108 equals the head tree fa0cb6e34b68dd70379645fc83222ad334c50108.

D122a floor: 67 forms (2% of 3,348). This batch gains +84 forms and +456 obligations, with 0 forms and 0 obligations lost (coverage diff against main be4dfac), so it clears the floor by 17. It also carries #39's checker changes: the not_submitted unobservable cause, [[reading]] grammar in each form's definition hash, and the optional receipt account_state.

kvlonge added a commit that referenced this pull request Nov 4, 2026
Basis: main 00e831a (PR #38, #39, #40), catalogue 5f35885,
evidence 469edde, parser tree 08415e1.

- 68 rebinds (ddl 1, expressions 6, functions 59, query 2): #39 put
  [[reading]] grammars into the definition hash; each fingerprint equals
  one an earlier full-read review approved.
- 149 newly approved forms (ddl 71, functions 56, scripting 14,
  expressions 5, query 2, transaction_session 1), read against their
  pinned pages and raw parser trees.
- Not approved: 37 forms with catalogue findings and 1 withhold
  (sf.ddl.drop-sequence: RESTRICT is dropped from the tree).

Checker at 00e831a: valid, 3476 forms; 1239 of 3348 verified.
@kvlonge
kvlonge deleted the fix/parser-gaps-b branch November 9, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant