Skip to content

perf(rules): evaluate built-in rules incrementally per model - #939

Merged
kvlonge merged 3 commits into
mainfrom
perf/incremental-builtin-rules
Oct 4, 2026
Merged

kvlonge merged 3 commits into
mainfrom
perf/incremental-builtin-rules

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 4, 2026

Copy link
Copy Markdown
Contributor

Why

One-model edits re-evaluated built-in rules with whole-project overhead: re-hashing the full native request, re-encoding every model for its cache key, and decoding one large JSON request in a single pass. Warm compiles hashed 60–190 MB of JSON just to find the stored response (CHI-441, profile item O7).

Changes

  • Every built-in rule declares a scope (model, project, both, or SQL lint); the engine rejects findings outside the declared scope, and a Rust test pins the classification.
  • Each model's payload is encoded and hashed on its own and keys that model's cached findings, with the rule set, rule config, dialect and native build. Suppression settings are applied after retrieval, so editing them keeps the model cache.
  • The whole-response memo is keyed by the request without models plus the ordered model hashes. Models decode in parallel on a miss; deleted models are pruned; a corrupt cache falls back to full evaluation.

Verification

  • Dense one-model edit CPU (base → head): 3k 11.0 → 10.0 s, 10k 34.2 → 29.9 s; peak memory 10k 1960 → 1712 MB. Output identical across 16 base/head pairs.
  • Oracles: incremental equals full evaluation over a 12-step edit sequence and a 10-edit real-CLI chain on DuckDB; corrupt-cache recovery.
  • One independent review; make test, make check-ci.

@kvlonge
kvlonge enabled auto-merge (squash) October 4, 2026 10:43
Declare an explicit scope on every built-in rule and reject findings a rule
emits outside its declared scope, so a new rule must state whether its
findings are model-local or cross-model.

Send each model to the native engine as its own encoded payload with a
digest of those exact bytes. The digest keys that model's cached findings,
the whole-response memo is keyed by the model-free request plus the ordered
digests, and models are decoded in parallel. Model findings no longer depend
on a project-wide fingerprint or on suppression lists, cross-model rules
always re-run, and cache entries for deleted model files are pruned.

Hash rule identities with BLAKE2b instead of SHA-256.
Decode the split native rules request into a native handle while the memo is
resolved, so the encoded model payloads are freed before built-in rules run
and the decoded request is dropped as soon as evaluation finishes.
The first model's built-in rule cache key used to include a project-wide
fingerprint, so any edit re-evaluated it with identical findings. Model keys
are now model-local, leaving one built-in miss and one SQL-lint miss for the
edited model.
@kvlonge
kvlonge force-pushed the perf/incremental-builtin-rules branch from 591925a to 9bb85e3 Compare October 4, 2026 15:05
@kvlonge
kvlonge merged commit 6ddd00b into main Oct 4, 2026
34 checks passed
@kvlonge
kvlonge deleted the perf/incremental-builtin-rules branch October 4, 2026 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant