Repository navigation
perf(compile): plan SQL tests in the background in every mode - #926
Merged
Merged
Conversation
Native SQL-test planning serialized each parsed model statement to a serde_json Value only to find its FROM and JOIN marker calls. A streaming serializer now reproduces that walk, including key order, repeated-key replacement and to_value failures, cutting native planning CPU by about 40 percent. A JSON-walk oracle in the tests pins the equivalence.
SQL-test planning only ran in the background for uncached projects of 128 to 5000 models with rules selected, so cache-on and large compiles planned on the critical path in the write phase. Prepared artifacts now fall back to planning SQL tests alone in the background, against the published target and its artifact cache, and the write phase consumes that plan at the point it would otherwise plan inline. Results and errors are identical to inline planning.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
SQL-test planning is the largest single CPU cost of a cold compile (6.7 s at dense 3k, 23.6 s at 10k). It ran in the background only for
--no-cachecompiles with rules selected and 128–5,000 models. With the cache on, above 5,000 models, or without rules, it ran inline on the critical path, so a cold compile with the cache was slower than--no-cache(CHI-441 phase 2, CHI-447).Changes
Verification
make test,make check-ci.