Skip to content

Read COPY INTO a location's PARTITION BY and parameter order as Snowflake does - #145

Closed
kvlonge wants to merge 15 commits into
mainfrom
fix/copy-partition-by
Closed

kvlonge wants to merge 15 commits into
mainfrom
fix/copy-partition-by

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #144 (its two commits come first), so it lands after #142, #143 and #144. This is the auditor's #144 finding: PARTITION BY with a parenthesized expression, and PARTITION BY followed by HEADER, failed to parse.

Commits after #144:

  • f37efb5 adds eight observed sf.dml.copy-into-location positives (o540–o547): three PARTITION BY shapes from the page's examples, and five that separate an external location's parameters from PARTITION BY.
  • dc6a9e9 is A's import 172 (run cpby-ex-00, 0.105 credits), with receipts for all eight.
  • df96345 settles them. Six parse and fail later (2003 or 90105), so they become accept, stage semantic. Two are syntax errors (1003): … FROM t1 STORAGE_INTEGRATION = i1 PARTITION BY 1 at BY, and … STORAGE_INTEGRATION = i1 FROM t1 … at STORAGE_INTEGRATION. Those become reject.
  • 9bad3d7 has the parser changes:
    • PARTITION BY takes a full expression. Before, the option reader took BY as PARTITION's value.
    • In Snowflake's faithful modes, anything but FROM right after the location is SQ0001 with a hint, and so is PARTITION BY after STORAGE_INTEGRATION, CREDENTIALS or ENCRYPTION. This also covers the six d91-dml-1 1003 receipts, which the parser used to refuse for the wrong reason.
    • A stage reference no longer takes FROM as its name (@~, @, @%), and it reads a table stage after a namespace (@v1.%v1). Five recorded cases parsed before only because FROM was then read as an option; they still parse.

Audit follow-up (4381cbb, e370a79):

  • COPY INTO a table has no PARTITION BY: COPY INTO t1 FROM @s1 PARTITION BY 1 is refused again (SQ0001 with a hint, faithful modes), as before 9bad3d7.
  • A bare @~, @ or @% no longer takes an option name such as FILES or PATTERN as its own name, so COPY INTO t1 FROM @~ FILES = ( 'a' ) parses. This gap predates the PR.
  • Four observed cases await receipts: PARTITION BY after CREDENTIALS and after ENCRYPTION (copy-into-location), PARTITION BY on a table, and FILES after @~ (copy-into-table). The diff is unchanged: 45 gained, 0 lost, 5,096 contradicted.

Tests: ten acceptance rows in test_snowflake_recorded.rs, which fail without the parser change.

syntax-coverage diff coverage/syntax/baseline.json at 9bad3d7 (parser tree 470c9ade):

  • Forms gained or lost: none. Obligations lost: none. stale is empty.
  • Obligations gained: 45, against Require a source after FROM or TO in COPY #144's 22. The 23 added here are all copy-into-location: o017, o021, o023, o024, o044, o053, o067, o092, o262, o508, o509, o510, o525, o531, o533 and o540–o547.
  • Receipts the parser contradicts drop from 5,115 to 5,096.

Checks: sqlquack-syntax and sqlquack-syntax-coverage tests, clippy -D warnings, fmt, fensu (scripting.rs at 1,995 lines) and the hygiene scan. The full gate runs at the slot.

A rebuild with the case builder settles ten cases whose receipts landed without a rebuild:
six observed cases take the expect their receipts agree on (three reject, three accept with
a semantic stage), and four accepted cases gain the semantic stage their receipts record.
No case is added or removed and no SQL, setup or session changes.
The source list stops before a name followed by =, taking it for the first copy option, so
COPY INTO <location> FROM v1 = ( ) parsed with no source and read v1 = ( ) as an option.
Snowflake rejects it as a syntax error (1003). An empty source after FROM or TO is now a
parse error; COPY INTO <table> with no FROM still parses.
The page's examples partition by a parenthesized expression, such as
PARTITION BY ('date=' || ...), and follow it with HEADER = TRUE, but no case covered either
shape. Three observed positives (o540-o542) ask Snowflake for its verdict.

Six recorded syntax errors write an external location's STORAGE_INTEGRATION or CREDENTIALS
and ENCRYPTION after FROM, then PARTITION BY, but no receipt says which part Snowflake
refuses. Five observed positives (o543-o547) separate the location parameters' position
from PARTITION BY.
cpby-ex-00, the eight copy-into-location cases built at f37efb5. Two are
syntax errors (1003/42000): PARTITION BY after STORAGE_INTEGRATION, and
STORAGE_INTEGRATION before FROM. The other six fail after parsing (2003 on
the missing table, 90105 with no current database). Inventory equal, no
changes, 0.105 credits.
Import 172 answers the eight observed COPY INTO <location> cases: six parse and fail later
(2003 or 90105), so they settle as accept with a semantic stage; PARTITION BY after
STORAGE_INTEGRATION and STORAGE_INTEGRATION before FROM are syntax errors (1003), so those
two settle as reject.
…lake does

- PARTITION BY takes a full expression, so ( 'date=' || v1 ) and a following HEADER = TRUE
  parse; before, the option reader took BY as PARTITION's value.
- In Snowflake's faithful modes, anything but FROM right after the location, and PARTITION BY
  after STORAGE_INTEGRATION, CREDENTIALS or ENCRYPTION, are syntax errors with a hint, as
  import 172 records (1003 at STORAGE_INTEGRATION and at BY).
- A stage reference no longer takes FROM as its name (@~, @, @%) and reads a table stage
  after a namespace (@v1.%v1); those parsed before only because FROM was then read as an
  option.
kvlonge added a commit that referenced this pull request Nov 7, 2026
… options after @~

Only PARTITION BY after STORAGE_INTEGRATION has a receipt. Four observed cases ask Snowflake
for the rest: PARTITION BY after an Azure location's CREDENTIALS and after an S3 location's
ENCRYPTION, PARTITION BY on COPY INTO a table, and FILES after a user stage source.
…re stage

- COPY INTO a table has no PARTITION BY. Reading it as an expression accepted
  COPY INTO t1 FROM @s1 PARTITION BY 1, which failed to parse before; in Snowflake's
  faithful modes it is now a syntax error with a hint.
- A bare @~, @ or @% no longer takes an option name such as FILES or PATTERN as its own
  name, so COPY INTO t1 FROM @~ FILES = ( 'a' ) parses.
The ENCRYPTION case also wrote STORAGE_INTEGRATION, whose receipt already decides it, so
its receipt could not say whether ENCRYPTION alone puts PARTITION BY out of place.
kvlonge added a commit that referenced this pull request Nov 7, 2026
…table, 3 receipts

cp145-ex-00, three of the four cases built at e99c160. All three are syntax
errors at BY (1003/42000): PARTITION BY after CREDENTIALS, after ENCRYPTION
alone, and on COPY INTO a table. The fourth (FILES after @~) was held by the
guard because it touches the connecting user's stage, and was not run.
Inventory equal, no changes, 0.0 credits.
…ess stages

Import 173 records 1003 at BY for PARTITION BY after an Azure location's CREDENTIALS
(o548), after an S3 location's ENCRYPTION alone (o549) and on COPY INTO a table (o726), so
all three settle as reject. The @~ FILES case (o727) stays observed: the receipt guard holds
the connecting user's stage offline. Two observed positives ask the same question of the
nameless stages @ and @% (o728, o729).
kvlonge added a commit that referenced this pull request Nov 7, 2026
…ipts

cp145b-ex-00, the two cases built at b19fdea. COPY INTO t1 FROM @ and FROM @%,
each with FILES = ( 'a' ), fail after parsing on the missing table
(1757/42601). Inventory equal, no changes, 0.219 credits.
Snowflake parses COPY INTO t1 FROM @ FILES = ( 'a' ) and @% FILES = ( 'a' ) and fails them
after parsing (1757), so o728 and o729 settle as accept with a semantic stage.
The empty-source check ran in every dialect and mode, so PostgreSQL COPY statements that
polyglot also refuses failed with a different message in the exact check. It now applies
only where Snowflake's statement-context rules do, as the other Snowflake refusals.
kvlonge added a commit that referenced this pull request Nov 7, 2026

kvlonge commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Closing: SQLQuack lands through local-gate trains, not PR merges (D94), so open PRs only add clutter (D628/D630). The branch is kept. Any work here that isn't on main yet is tracked by the train owner and lands through a train.

@kvlonge kvlonge closed this Nov 8, 2026
@kvlonge
kvlonge deleted the fix/copy-partition-by branch November 9, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant