Repository navigation
feat(syntax): check Snowflake table DDL against its grammar - #179
Merged
Merged
Conversation
CREATE EXTERNAL TABLE was checked against its grammar but kept as raw text, so every accepted statement counted as an opaque fallback. It now returns a CreateObject with column definitions, computed expressions, inline constraints and properties, and the grammar still refuses the same statements.
A CREATE TABLE the table model cannot hold, such as one with column masking or projection policies or table policies, was checked against its grammar and kept as raw text. It now becomes a CreateObject of kind TABLE: columns with their options, out-of-line constraints and table properties as properties. The shared name, name list, tag and value checks now return what they read.
…as nodes CREATE DYNAMIC ICEBERG TABLE and CREATE INTERACTIVE TABLE were checked against their grammar and kept as raw text. They now become a CreateObject with their columns, clustering, properties and the query or REFRESH USING insert.
… tables These kinds fell through to the generic CREATE, which kept any text as raw, so malformed statements were accepted. They now have grammars that build a CreateObject: an event table takes table properties, an event routing table its rules and an online feature table its key, properties and source. Partial kinds such as CREATE FEATURE TABLE are refused, and a table property may now be written WITH COMMENT = <value>.
…r grammar CREATE ICEBERG TABLE and CREATE DYNAMIC TABLE went through the generic table parser, which refused valid column options and properties and accepted properties with no value. In the faithful Snowflake modes they are now built by the table grammar with each kind's properties: an iceberg table may have no columns, partitions only over its columns and takes no OR ALTER; a dynamic table needs TARGET_LAG, WAREHOUSE and a query or REFRESH USING. TRANSIENT now reaches both kinds, and a table the generic parser finishes with raw properties is built by the table grammar when that holds.
… grammar DEFINE TABLE and DEFINE DYNAMIC TABLE were refused outright, because the generic table parser behind them was not checked closely. They are now built by the table grammar like their CREATE OR ALTER forms.
LOCAL and GLOBAL before TEMP or TEMPORARY, and VOLATILE before TABLE, sent the statement to the generic CREATE, which kept it as raw text. They now mark a temporary table. The table grammar also takes a constraint COMMENT and a GENERATED ALWAYS AS ( <expr> ) [ VIRTUAL ] column.
The tag, contact and policy actions of ALTER TABLE, ICEBERG TABLE, VIEW and MATERIALIZED VIEW were checked against their grammar and kept as raw text. They now become an AlterObject: the action's words, the added columns or dropped policy as operands, and the tags, contacts and policies as properties, one per column for a column action.
ALTER ICEBERG TABLE ADD PARTITION BY and DROP PARTITION BY were refused as if they were the external-table partition actions, and a transform with no arguments, such as YEAR ( ), was refused in every partition clause. Both actions now take one partition expression like REPLACE PARTITION BY, and a transform may be written with no arguments.
An added column now takes the CREATE TABLE column options, including constraint properties such as NOT ENFORCED or ENABLE VALIDATE, and must have a type. An out-of-line constraint after ADD takes the same properties and keeps them in order on the constraint. A constraint without ADD and ADD ( <column> ) are refused.
…constraint renames ALTER TABLE takes DROP KEY for the clustering key, DROP UNIQUE and DROP FOREIGN KEY with an optional column list, each with CASCADE or RESTRICT, and RENAME CONSTRAINT <name> TO <name>. A key or primary key still takes no column list.
ALTER [ ICEBERG ] TABLE ADD and DROP SEARCH OPTIMIZATION read their targets with the dynamic table grammar and build an AlterObject: ON needs a target, a list of targets is one action, and a reserved word is refused.
…ctions ALTER COLUMN takes SET DATA TYPE <type> RENAME FIELDS (or a bare RENAME), [ SET ] WRITE DEFAULT <expr> and DROP WRITE DEFAULT. SET DATA needs TYPE. DROP VERSION_TAG with nothing after it drops a column of that name.
ALTER [ ICEBERG ] TABLE takes RECLUSTER [ MAX_SIZE = <n> ] [ WHERE <condition> ], SUSPEND RECLUSTER and RESUME RECLUSTER, and ALTER ICEBERG TABLE takes REFRESH [ '<metadata path>' ], each as an AlterObject.
ALTER TABLE SET takes <property> = <value> pairs separated by spaces or commas, keeping each as a property, and refuses SET with no property name.
ALTER TABLE refuses DROP PARTITION BY, which only an iceberg table takes, and MODIFY DATA FUNCTION, MODIFY DATA METRIC or MODIFY METRIC FUNCTION without the full DATA METRIC FUNCTION. ALTER ICEBERG TABLE refuses a RENAME without COLUMN or TO.
A hybrid table's INDEX needs a name and a column list and may take INCLUDE ( <columns> ), which the index keeps. A hybrid table the table parser cannot finish, such as one whose comment is a name, is read with the CREATE TABLE grammar.
EXECUTE AS USER <user> USE SECONDARY ROLES takes a comma-separated list of roles, not only one.
ALTER SESSION SET refuses a setting with no value and a SET with no setting, and reads settings separated by spaces as well as commas.
TO_TIMESTAMP ( ) and TRY_TO_TIMESTAMP ( ) parse, as Snowflake's parser takes them; their arity is a later check. The converter already refuses them as calls without an argument.
… grammar The table grammar satisfies 2,899 more obligations with no coverage regression; re-record the baseline so check-syntax holds the gain. Eight corpus statements are now refused, as Snowflake refuses them: ALTER TABLE ADD with a parenthesised column, a MySQL index prefix length or column position, a UNIQUE constraint with no column list, and a dynamic table's column list after its properties.
The table grammar built column and external names without their token's range, so a refused ALTER TABLE ADD COLUMN pointed only at the table name. Names now keep the range of the token they were read from.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The ddl table forms had the largest parser-side gap: opaque fallbacks, false rejects and false accepts in CREATE and ALTER of tables. This replaces their Raw fallbacks with checked grammars that reject what Snowflake rejects.
Changes
One commit per root cause:
Verification
Snapshots on evidence b8c72a0, main's parser → this head:
syntax-coverage diff BASE HEAD: 0 regressions, 2,899 obligations gained; seven documented iceberg REST cases are now accepted.In the 124 table forms (6,569 judged cases): opaque 800 → 0, false rejects 434 → 3, false accepts 223 → 1. Open: a computed column with LOCATION in CREATE TABLE (2), an unnamed reference in DEFINE TABLE (1), ADD REFERENCES v1 ( SELECT ) (1).
Still unjudged: the iceberg table clone case (receipt pending), and ALTER TABLE UNSET … CHANGE_TRACKING o026 (awaiting observation) and o029 (its reject expectation conflicts with its receipt).
make check-cipasses; crate tests cover each root cause.