Repository navigation
Conversation
…rties in Snowflake trees DEFAULT in INSERT VALUES and multi-table intoClause VALUES is a default_keyword node, not a variable or column; ALTER DYNAMIC TABLE MODIFY <col> COMMENT / UNSET COMMENT is an alter_column node; inline RELY/NORELY are kept with the constraint they follow, and a named column-level FOREIGN KEY keeps its constraint name.
…orded grammar A token check before COPY is read refuses what Snowflake refuses: missing INTO or FROM, strings or spaced quoted names as stage parts, reserved words as names, a load's query without a stage source, empty or keyless option values, and PARTITION without BY. A Snowflake COPY that passes the check but that the typed reader cannot read in full (COPY FILES, PARTITION BY, metadata string keys, empty stage name parts) is kept as a command.
…ATCH SKIP TO takes a bare symbol WIP, not yet tested: a Snowflake interval unit is never a call, and NEXT without ROW is the symbol of TO <symbol>.
…wflake NEXT lexes as a keyword, so the earlier check for a NEXT variable never matched and TO NEXT without ROW fell through to expect_identifier, which refuses the keyword. The pinned test now passes.
…S, MATCH_RECOGNIZE, SEMANTIC_VIEW Brings 45 of the query family's 46 recorded disagreements into agreement, with no regressions across the recorded syntax cases. Still open: FINAL inside DEFINE.
…e commands Brings all 54 of the DML family's recorded disagreements into agreement, with no regressions across the recorded syntax cases. MERGE takes UPDATE and INSERT ALL BY NAME and requires INTO and MATCHED; DELETE requires FROM and a query in USING ( ); PUT, GET, LIST and REMOVE refuse spaces in unquoted locations and quoted names.
CREATE SEMANTIC VIEW was kept as written whatever followed the kind, so every malformed one was accepted. It is now checked clause by clause (TABLES, RELATIONSHIPS, FACTS, DIMENSIONS, METRICS, AI_* clauses, COMMENT, COPY GRANTS, TAG) before it is kept, which brings 395 of the DDL family's recorded disagreements into agreement with no regressions. A metric window may PARTITION BY EXCLUDING only inside this check.
The general CREATE TABLE path refused every column-form external table. External tables now have their own check (virtual columns with inline constraints, PARTITION BY, a stage LOCATION, FILE_FORMAT and the other properties, USING TEMPLATE of a query, a string or a variable) and are kept as written. This brings 611 more of the DDL family's recorded disagreements into agreement, with no regressions.
…ACTIVE TABLE against their grammar Both forms used to fall through to the raw fallback, so any token run after the kind was accepted. A grammar check now runs first and keeps the statement as written only when it holds: a name that is not a string or reserved word, typed or untyped columns with COMMENT, masking policy and tag options, and the properties the recorded receipts accept. A dynamic Iceberg table needs TARGET_LAG, WAREHOUSE and AS <query> or REFRESH USING ( INSERT ... ); an interactive table needs CLUSTER BY and AS <query>. PARTITION BY, row access, aggregation and join policies on interactive tables, OR REPLACE with IF NOT EXISTS, and INTERACTIVE without TABLE, WAREHOUSE or MATERIALIZED VIEW are refused.
…s accept A return type may be followed by NOT NULL or NULL, also after RETURNS TABLE ( ... ). COPY = ( ... ) is a name list and may be empty. HANDLER, TARGET_PATH, COMMENT and IMPORTS items take a bare word as well as a string; SECRETS keys may be bare words; SERVICE, API_INTEGRATION, EXTERNAL_ACCESS_INTEGRATIONS items and SECRETS values may be strings. MEMOIZABLE is an option. A RETURNS TABLE column may omit its type. Missing values, unclosed lists and NULL ON NULL INPUT stay refused.
…against their grammar A Snowflake database or schema took none of its properties, so even COMMENT = 'm' was refused. The properties after the name, and after a clone's IGNORE and INCLUDE options, are now read: <key> = <value> with a literal, quoted name or unreserved word, [WITH] TAG ( ... ), WITH CONTACT ( ... ) whose values are names, WITH MANAGED ACCESS on a schema, and LINKED_CATALOG = ( ... ) and FROM LISTING on a database. A missing value, a reserved word, WITH ( ... ), and a property after TAG or CONTACT are refused, as the recorded receipts are.
…es as written The table model holds no masking, projection, join, aggregation or storage lifecycle policy, tag, contact, copy option or constraint property. When the table parser cannot finish a Snowflake CREATE TABLE, the statement is checked against Snowflake's CREATE TABLE grammar and kept as written if it holds, so translation refuses it instead of dropping what it cannot carry. Statements the table parser finishes are unchanged.
… lists Snowflake rejects MATCH without a match type, NO without ACTION, INITIALLY without DEFERRED or IMMEDIATE, a constraint COMMENT without a string, a primary key on no columns, a reserved word as a tag name and a parenthesized property list after a table's WITH; the parser accepted each of them.
… their grammar An ALTER TABLE, ICEBERG TABLE, VIEW or MATERIALIZED VIEW whose action sets or unsets a tag, contact, masking, projection, row access, aggregation or join policy, or adds a column carrying one, is checked against Snowflake's grammar before the generic ALTER parser and kept as written, so translation refuses it rather than dropping the policy. Malformed policy actions the generic parser accepted are now refused; a materialized view takes no contact purpose. The generic parser already kept these actions as raw fragments; the statement is now raw as a whole. Also accept ALTER ICEBERG TABLE CONVERT TO MANAGED, a session variable after USING TEMPLATE, and TARGET_LAG and WAREHOUSE among CREATE TABLE properties.
… list Snowflake takes ALTER TABLE ... ALTER followed by a parenthesized list of column actions, and a column default without SET. Each listed item becomes the same AlterColumn action as the unparenthesized form; a list left open or an item the unparenthesized form refuses is refused.
Column UNSET TAG and UNSET MASKING POLICY need COLUMN, an added column takes one policy or tag list, VALIDATE and NOVALIDATE follow ENABLE or DISABLE, a Java, Scala or Python procedure takes no null handling or volatility, an unload's PARTITION BY comes before its other options, and an external stage quoted with a blank is refused. External table columns Snowflake parses before refusing (a bare name, a bare REFERENCES) parse again.
… only in faithful modes A governance ALTER that passes its grammar check keeps the normal ALTER tree when that tree covers the same tokens and no name after the object is quoted; otherwise it stays Raw, since the tag and row access policy actions drop the quotes. A bare USING TEMPLATE ( query ) or 'text' external table keeps the structured CREATE TABLE. The Snowflake verdict rules for TABLE( ) around FROM calls, MERGE INTO, WITH table properties, interval units, sample sizes and external tables apply in the faithful modes only, like the other statement-context rules, so the lowered polyglot comparison parses as before. The exact check lists the PUT assertion-message lines and table UNSET PROJECTION POLICY (recorded 1003) as expected rejections.
…h (D303) Moves out of met, as ruling D303 accepts: six quoted-name governance obligations whose structured tree dropped the quotes and now stay Raw (alter-iceberg-table.datagovnpolicytagaction o243, o245, o247 and alter-table.datagovnpolicytagaction o247, o249, o251), and sf.dml.insert, unverified until its review is renewed at the new fingerprint. Everything else moves into met: 1,487 obligations gained, 1,040 syntax-error cases now rejected, 1,022 refused cases now parsed and 2,400 receipt contradictions resolved.
Contributor
Author
|
D94 gate: head 552b9c1, command |
Only change: sf.dml.insert unverified -> verified (auditor-dml-004 at fingerprint cd9bb886), back to 2,388 verified forms; every other snapshot field is identical.
Contributor
Author
|
Auditor approved 552b9c1 (record 80bbed8 on wip/syntax-omission-audit): D303 list of 6 confirmed independently, counts match. Fast-forwarded the auditor's INSERT re-review 6a08b06 (auditor-dml-004 at fingerprint cd9bb886) and re-recorded the baseline in 1b2354f: only sf.dml.insert unverified -> verified. |
kvlonge
changed the title
Only the parser tree id changes, to the merged Snowflake parser. The diff against the committed baseline and against main's lists no obligation lost beyond the six quoted-name obligations the batch already gives up, and gains 1,487.
kvlonge
added a commit
that referenced
this pull request
Nov 7, 2026
… prefixes The merge left the Snowflake statement entry file at 2002 lines, over the 2000-line limit. parse_with moves unchanged to with_statement.rs, which already parses the statement a WITH clause prefixes.
Only the parser tree id changes; every claim is identical.
…er types Only the parser tree id changes; every claim is identical.
Contributor
Author
|
Landed on main as squash commit 3115fa3, a fast-forward from 048e7dd. Its tree, b15fc9579fee1211be591927220403e8181feb47, is the tree of the gated head. Gated head: 8c237f7 (wip/parser-snowflake-a-b2, main 048e7dd merged in), CARGO_BUILD_JOBS=1:
Syntax coverage against main's baseline: 1,487 obligations gained, 6 lost (the quoted tag, policy and column name obligations in ALTER TABLE and ALTER ICEBERG TABLE governance actions, accepted in review), 0 forms lost. Hygiene scan clean. The landed commit message carries no tracker IDs, so this lands as a squash rather than the branch history. |
Contributor
Author
|
Closed: landed as squash 3115fa3 on main (see the comment above). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Snowflake parser batch from lane A (D133 query, DML and DDL slice): grammar checks for COPY, CREATE EXTERNAL TABLE, dynamic iceberg and interactive tables, semantic views, database and schema properties, CREATE TABLE policies and properties, ALTER tag, contact and policy actions, the parenthesized ALTER ( column action, … ) list, and query and DML verdicts. Each refusal is backed by a recorded Snowflake receipt.
Gate (D94)
make check-merge RECORDER=0on head552b9c1b69a9e76a4dfe1b1f64308e00e01444d4(main f2d93ee merged in), CARGO_BUILD_JOBS=1: exit 0. That covers fmt, clippy in both configurations, ruff, ty, fensu, syntax-coverage diff, the evidence and polyglot guards, generated and referee cases, the python API and all 24 Rust test binaries. Separately,syntax-coverage summaryexits 0, andsyntax-coverage stalelists onlysf.dml.insert(below). The hygiene scan is clean.Main has moved since then (fc15d51). Per D94, this will be merged with the latest main and the gate re-run on that exact head before the fast-forward.
Coverage diff against main's baseline
syntax-coverage diff coverage/syntax/baseline.json, taken before re-recording:sf.dml.insert, stale review only (see below)Full-corpus verdict regression check (59,963 recorded Snowflake statements, base and head binaries built in separate target dirs and confirmed to differ): 2,422 newly agree with Snowflake, 0 regressed.
The re-recorded baseline moves nothing else out of met. Gains: 1,040 syntax-error cases are now rejected, 1,022 refused cases now parse, and 2,400 receipt contradictions are resolved.
summary: 2,387 of 3,351 forms verified (main 2,388; the one fewer issf.dml.insert), obligations 70,066 of 91,134.D303: obligations moved out of met
Ruling D303 accepted the governance losses where main's structured tree dropped information. This PR now keeps main's structured tree whenever it is faithful: it re-parses through the normal ALTER path and keeps that tree when the tree covers the same tokens and no name after the object is quoted. A bare
USING TEMPLATE ( query )external table also keeps the structuredCREATE TABLE. That brings back 161 of the 167 losses first reported. The 6 that remain all come from one group, a quoted identifier inside a tag, row access policy or ON column list:sf.ddl.alter-iceberg-table.datagovnpolicytagaction.o243(quoted<tag_name>)sf.ddl.alter-iceberg-table.datagovnpolicytagaction.o245(quoted<policy_name>)sf.ddl.alter-iceberg-table.datagovnpolicytagaction.o247(quoted<col_name>)sf.ddl.alter-table.datagovnpolicytagaction.o247(quoted<tag_name>)sf.ddl.alter-table.datagovnpolicytagaction.o249(quoted<policy_name>)sf.ddl.alter-table.datagovnpolicytagaction.o251(quoted<col_name>)Condition 1: main's tree loses the quoting.
crates/sqlquack-syntax/srcis unchanged between d19c7a5, where the trees were probed, and main. The trees below are from that parser:ALTER TABLE v1 SET TAG "v1 Q" = 'm'SetTag { expressions: [("v1 Q", 'm')] }: a bare string, with no quoted flagRawALTER TABLE v1 ADD ROW ACCESS POLICY "v1 Q" ON ( v1 )Raw { sql: "ADD ROW ACCESS POLICY v1 Q ON(v1)" }: the quotes are goneRawstatement, text as writtenALTER TABLE v1 ADD ROW ACCESS POLICY v1 ON ( "v1 Q" )Raw { sql: "ADD ROW ACCESS POLICY v1 ON(v1 Q)" }Rawstatement, text as writtenALTER ICEBERG TABLE v1 …, the same threeRawThe unquoted spellings stay structured (
SetTag,alter_table), and tests pin both sides.Condition 2: Raw is refused.
sqlquack convertrefuses the Raw statements withSQ0002 missing_rule, "no rule converts node:raw as a query". On main the same statements are refused asnode:alter_table(ornode:create_table), so neither converts and the output does not change. Lineage resolves only query nodes. Any other node, Raw or ALTER, yields an open result with no outputs, so Raw is never counted as success.Condition 3: no obligation is retired or reworded (
catalogue_files_changed: 0, obligations files untouched).Condition 4: the 6 ids are listed above and sent to the Snowflake auditor for independent confirmation. The baseline is re-recorded in 552b9c1.
sf.dml.insert
The DEFAULT-in-VALUES tree fix changes its fingerprint to
cd9bb8864e68b288d7dcbef3012f13d970779fb739f18105ca0d0495b22e73b0, so reviewauditor-dml-003(recorded atca16c15e…) is stale. It stays Verified-pending until the auditor renews it, and the baseline records it as unverified.Polyglot exact check
The Snowflake verdict rules for
TABLE( )around FROM calls,MERGEwithoutINTO, tableWITHproperties, interval units, sample sizes and external tables now apply only in the faithful modes, like the other statement-context rules. The product (FaithfulLocated) and coverage (Faithful) verdicts are unchanged, and the lowered polyglot comparison parses as before. New expected rejections are listed inexpected_differences.toml: the PUT assertion-message lines in polyglot's tests, andALTER TABLE t UNSET PROJECTION POLICY(recorded 1003). All three Snowflake corpora report 0 differences.Not merging yet
This PR joins the queue after the Postgres steps (D289). It waits for the product lane's go-ahead and the auditor's confirmation of the D303 list and the insert review.