Repository navigation
fix: keep authored SQL tokens when formatting - #846
Merged
Merged
Conversation
sqb format regenerated SQL from the parse tree, so any spelling the SQL library canonicalised was written back into the user's file. In Snowflake that renamed STARTSWITH to STARTS_WITH, which Snowflake rejects, and also rewrote SUBSTR, TIMESTAMPDIFF and TRY_TO_DECIMAL to synonyms. The formatter's syntax-tree check could not see this because both spellings parse to the same node. After generating the layout, walk the authored and generated token streams together and write the authored text of every aligned token into the new layout. Only whitespace, the case of unquoted words, an added AS, and dropped optional terminators may differ; any other structural difference refuses the format and leaves the file unchanged. The same check runs on the idempotence pass. On a 1,337-file Snowflake project this removes all 82 function renames, with no new refusals.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
sqb formatregenerated SQL from the parse tree, so canonical spellings from the SQL library were written back into user files. In Snowflake it renamedSTARTSWITHtoSTARTS_WITH, which Snowflake rejects with "Unknown function", and rewroteSUBSTR,TIMESTAMPDIFFandTRY_TO_DECIMALto synonyms. The formatter's syntax-tree check could not see this, because both spellings parse to the same node.Changes
AS, and dropped optional terminators may differ. Any other structural difference refuses the format and leaves the file unchanged with a named reason. The idempotence pass uses the same check.STARTSWITH/SUBSTR/TIMESTAMPDIFF/TRY_TO_DECIMALand DuckDBIFNULL/!=, and all of them fail on the previous formatter.Known limitation, tracked in CHI-402: PostgreSQL
substring(a, 1, 2)andSELECT ALLare now refused rather than restructured.Verification
make test: 7,794 passed.make check-ci: passed, including clippy on Rust 1.98.