Repository navigation
perf(format): format files in parallel and skip unchanged files - #1183
Merged
Merged
Conversation
…and tokenization Format runs now hold file texts in Rust between passes instead of round-tripping every file through Python dictionaries, reuse each text's header lint for the final report, skip per-line keyword allocations in the header scanner, and reuse token and oracle lexemes across the formatter's verification and idempotency passes.
Header lint, header formatting, body preparation and splicing, rewrite checks, suppression scanning and file writes now run across a format worker pool with 16 MiB stacks. Results are collected in file order and the first failure in file order is reported, so output, ordering and exit codes do not depend on scheduling. Python work (fixture contracts, Rule fixes, description lookups and report records) stays on the calling thread.
sqb format now remembers every file it found already formatted with nothing to report, in target/cache/compiler-native-v1/format-results.bin. Entries are keyed by the file's project-relative path, its exact text, the effective format and rule threshold settings and dialect, and the SQLBuild build, so an edit, a settings change or another build formats the file again; a damaged store opens empty. --fix never uses the cache, SQL test files are not skipped while fixture-null formatting reads model contracts, and the new --no-cache flag formats every file.
…ailure Formatting stays parallel, but files are replaced in path order again and a failed write leaves every later file as authored. The --no-cache help text now matches the other format flags.
…ath order Replacement files are written and synced beside their targets in parallel, then renamed in path order. The earliest file whose staging or rename fails stops the run: it and every later file keep their text, and every unused replacement file is removed.
kvlonge
enabled auto-merge (squash)
October 20, 2026 22:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
sqb format --checkon a 10,000-model project (12,690 files) took 21–25 s and about 324 G instructions. Header work ran serially on one thread, and every run formatted every file again, so pre-commit and CI paid the full cost even when nothing had changed.Changes
target/cache/compiler-native-v1/format-results.bin. The key covers relative path, text, format settings, dialect and build.--fixand fixture-dependent test files bypass the store;--no-cacheformats every file.Verification
--check: 324 G → 240 G instructions, 21–25 s → 10–11 s wall.--check,--diffand--json, cold and warm, are byte-identical to base on the 1k and 10k projects and a fault-heavy variant. Written trees of all example projects are also identical.--no-cache, serial vs parallel output, and a failed write leaving later files untouched.make compiler-differentialgives 373/373, andmake check-cipasses.