Skip to content

perf: remove redundant graph-phase scans and parallelise source and resource SQL preparation - #1137

Merged
kvlonge merged 5 commits into
mainfrom
perf/parallel-graph-loops
Oct 18, 2026
Merged

kvlonge merged 5 commits into
mainfrom
perf/parallel-graph-loops

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

Why

The graph phase of a cold dense 10k compile (about 14 s) spent long stretches on the main thread doing work that scanned every model for every model, or recomputed values nothing read, while other cores idled.

Changes

  • Hash-map lookups replace per-model linear scans in the target duplicate check, contract diagnostics, SQL test grouping and audit shape lookup; first/last-wins order is preserved.
  • The set-operation keyword check compares whole words case-insensitively; declaration splicing has an ASCII fast path; the shape digest is computed only when read.
  • Source expressions are parsed ahead of the model loop in parallel off the GIL and reused only when the expanded SQL is identical; resource SQL preparation runs on the analysis pool. The first error is still reported in the original order, including when one item panics, and the pool is only created when there is resource SQL to check.

Verification

  • Compiled SQL and compile JSON identical on the 1k identity check.
  • Dense 10k, 5 paired cold runs: graph_ms median 13.7 s to 12.8 s, wall 26.9 s to 25.6 s, CPU 58.4 s to 57.3 s; instructions 426.5 G to 413.9 G.
  • Focused cargo tests (analysis session, semantic checks, compiled project, analysis cache, splice, set operations), new integration cases for source-expression error order, and the focused contract, semantic-check, SQL test and model analysis integration tests pass.
  • Independent review: no blockers; the pool-creation and panic-ordering notes are fixed.
  • make check-ci passes.

@kvlonge
kvlonge enabled auto-merge (squash) October 18, 2026 03:55
@kvlonge
kvlonge merged commit 4505bbb into main Oct 18, 2026
43 checks passed
@kvlonge
kvlonge deleted the perf/parallel-graph-loops branch October 18, 2026 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant