Skip to content

perf(compile): batch analysis preparation normalization - #957

Merged
kvlonge merged 2 commits into
mainfrom
perf/batch-analysis-preparation
Oct 6, 2026
Merged

kvlonge merged 2 commits into
mainfrom
perf/batch-analysis-preparation

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Why

Analysis preparation called native SQL normalisation once per model or shared binding query from Python. Most of that time was threads waiting to reacquire the GIL after each short call: about 1.6 s at dense 3k and 5.4 s at dense 10k cold (CHI-509).

Changes

  • The three normalisation points in analysis preparation now make one native call per chunk of 256 models with the GIL released. Each model keeps its own result or error in place, and Python raises errors at the same point and with the same message as before.
  • Chunks run in parallel on the compile's existing analysis pool when no analysis batch is running on it, otherwise sequentially in the same call, so preparing the next batch never waits behind a running one.

Verification

  • Quiet host, alternating, medians: analysis preparation at dense 3k cold 1597 → 834 ms (wall 17.35 → 16.54 s), dense 10k cold 5436 → 2947 ms (wall 53.2 → 51.9 s); warm-miss also faster; edits and a ~1,400-model project unchanged; peak memory within noise.
  • Oracle comparing per-model and batched preparation (payloads, cache keys, cache hits, compile output) on fixtures, random projects with errors and dense 120/3k; fails when a batched result is perturbed. Rust tests for ordered results and errors.
  • One independent review; make test, make check-ci.

Analysis preparation normalized each model's stubbed SQL with its own
native call, releasing and reacquiring the GIL once per model. Under the
analysis worker threads most of that time was spent waiting to reacquire
the GIL rather than normalizing.

Normalize every batch member in one detached native call instead. A
compile catalog runs the batch in parallel on its analysis pool while no
analysis batch occupies the pool, and in turn otherwise, so preparing the
next ready batch never queues behind a running one. Each member keeps its
own result or error in place, so the same first error is raised at the
same point as before, and prepared payloads, cache keys, and cache hits
are unchanged.
Normalize each preparation batch in chunks of 256 members so one native
call never holds copies of a whole dense batch at once. This keeps warm
compile peak memory at its previous level while preserving the single
GIL release per chunk.
@kvlonge
kvlonge enabled auto-merge (squash) October 6, 2026 01:47
@kvlonge
kvlonge merged commit 9a98b2f into main Oct 6, 2026
34 checks passed
@kvlonge
kvlonge deleted the perf/batch-analysis-preparation branch October 6, 2026 02:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant