Repository navigation
perf(format): remove repeated work from the native SQL formatter - #1188
Merged
Merged
Conversation
…ds one Protecting SQL for the layout parse no longer formats a sentinel name and upper-cases every token, and looks cast ranges up by start token instead of scanning them for each token.
A body the formatter leaves unchanged would be re-printed from the same text, tokens and layout oracle, so the re-print can only reproduce it. Changed bodies are still re-printed and refused when not idempotent.
Restoring fragments after the layout parse indexed every token of the generated SQL by its lower-case text. Only tokens that start with the sentinel prefix can match a fragment, so only those are indexed.
Aligning authored tokens with the layout guide built a formatted key string, with an upper-case copy for words, for every token. Lexemes now carry a small key kind and compare their raw text directly, ignoring ASCII case for words, which matches exactly what the strings compared.
The token check read the authored and printed token texts twice each, once for the case-folded comparison and once for keyword recases. The printer already holds the authored texts and fold flags, so the check reuses them, compares them with the printed tokens without upper-case copies of the authored side, and reads the printed texts only when a keyword recase must be confirmed.
Laying out CTE macros tokenized the restored layout guide again for every body. A CTE macro token starts with the lint sentinel prefix, so the pass now returns at once when the text does not contain it.
Formatting headers scanned the file for header spans again after moving leading comments. When trimming and relocation changed nothing, the spans scanned from the original text are reused.
Printed text identical to the authored body tokenizes to the authored tokens, so the token check compares the body with itself and can only pass. The check and the printed-text tokenize now run only for bodies the printer changed.
A line comment after the last token was printed with a newline after it. Format runs trim a body's trailing whitespace and keep it apart, so each run added that newline again and the file never settled. The printer now ends the output at the comment, as the authored body did.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A cold
sqb format --check --no-cacheon a synthetic 10,000-model project spends about 78% of its instructions in the native SQL formatter, much of it repeating work on identical inputs. Separately, a body ending in a line comment gained a newline on everysqb formatrun, so--checknever passed on such files.Changes
Verification
--check,--diff,--check --jsonand written trees are byte-identical to the previous build on 1k/10k projects (formatted, unformatted, fault-heavy) and all examples, except the one file the newline fix settles. Formatting every file twice changes nothing.cargo test -p sqlbuild-rules, format unit/integration/E2E tests and performance guards,make check-ci.--no-cacheinstructions: 10k formatted 237.0G → 148.2G (−37%), 10k unformatted 240.2G → 199.1G (−17%). Warm runs unchanged.