Repository navigation
perf(rules): cut custom-rule host overhead - #981
Merged
Merged
Conversation
Hosts now start before the parent writes the project payload and wait for it to be published, so interpreter start-up overlaps the payload encoding. A failed write publishes an empty payload so waiting hosts exit promptly. Hosts load the project with cyclic collection paused and then freeze it, and the rule-test harness is imported only when a test calls it, so rule modules no longer pull the compile pipeline into every host.
… read The host payload drops model fields and project state that no Rule fact exposes, such as the second authored query copy, lineage columns, and the scope index, so the parent encodes and every host decodes less. Model fields are dropped while the payload is written, after hosts have started, so runs whose custom results are all cached pay nothing extra.
Reading a regular, non-symlink file resolves its directory once per tree and checks the file itself with one lstat. Symlinks, missing files, and directories still take the full resolution, so outcomes and errors match.
Whole-project facts such as project.sources repeat objects shared between resources: every source declared in one file carries that file's contents and entries. Digests of these facts now encode each compiler dataclass once and refer to it by content digest, so the digest grows with distinct content instead of with repetitions. Model and project-tree facts keep the digest a host observes. A real-CLI edit chain with model- and project-subject custom rules split across hosts compares every warm compile with a cache-free oracle, and the oracle is shown to fail when source digests are deliberately stale.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The custom-rule phase spent most of its time on overhead rather than evaluating rules: starting the host, encoding and decoding the project payload, resolving project-tree reads, and digesting whole-project facts again for every recorded read. This change makes the phase faster without changing which findings are reported or their order.
Changes
read_textresolves through cached directory listings.Verification
--no-cache. A deliberately broken invalidation makes it fail.make test,make check-ci.