Skip to content

perf: scale declaration scope resolution with many scoped macros - #782

Merged
kvlonge merged 1 commit into
mainfrom
perf/declaration-visibility-scaling
Sep 23, 2026
Merged

kvlonge merged 1 commit into
mainfrom
perf/declaration-visibility-scaling

Conversation

@kvlonge

@kvlonge kvlonge commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Why

Warm compiles slowed sharply as projects added folder-scoped macros. A generated 3,000-model project with 500 used macros took about 33s to recompile unchanged, versus about 6s with 37 macros. Macro execution was negligible (499 and 2,999 call sites cost the same). The cost came from declaration-scope resolution: every SQL test was granted every macro its expected model could see, about 529,000 grant records. Those grants were then rescanned quadratically, sorted, and turned into per-test declaration contexts.

Changes

  • Expected-model macro grants cover only macros that the test SQL calls, including calls nested inside macro arguments. Constant and enum grants are unchanged.
  • Visibility resolution groups each resource's grants once instead of rescanning them per declaration.
  • Shared model visibility is computed once per directory, not once per test.
  • Scope identity sort keys are cached. The scope metadata projection reuses that cache.
  • The fresh-process cache guards (3k/5k/10k) now use one folder-scoped macro per six models, all of them used. Macro scaling is therefore checked at every size without an extra CI case.

Compiled SQL, diagnostics and compile JSON are unchanged. Scope metadata no longer lists grants for macros a test never calls.

Verification

  • 500-macro project, paired fresh-process runs: warm 33s → 6.5s and peak RSS 862 → 442 MiB, with identical semantic fingerprints.
  • A private real-project paired comparison showed no regression, with identical artifacts in all runs.
  • New unit cases: a test is granted only the macros it calls, and nested argument calls stay granted. Each fails without its fix.
  • Focused scope, SQL-test, scenario, macro-context and scope-CLI suites pass. make test passes.
  • The previous 3k guard fingerprints were unchanged before the guard shape changed.

@kvlonge
kvlonge enabled auto-merge (squash) September 23, 2026 05:40
@kvlonge
kvlonge merged commit eb21a7a into main Sep 23, 2026
28 checks passed
@kvlonge
kvlonge deleted the perf/declaration-visibility-scaling branch September 23, 2026 05:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant