Skip to content

perf: skip redundant SQL normalisation and CTE scans during compile - #1088

Merged
kvlonge merged 3 commits into
mainfrom
perf/engine-fixes
Oct 12, 2026
Merged

kvlonge merged 3 commits into
mainfrom
perf/engine-fixes

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Why

Profiling a generated 10,000-model project showed compile work that does not change output: every analysis-cache hit normalised its SQL eagerly although only a few readers need it, and SQL-test planning re-tokenised the same model SQL for every test and comparison body (about 7% of instructions on a 1,000-model cold compile).

Changes

  • Analysis-cache hits normalise their cleaned SQL only when read: batched before binding revalidation and case-sensitive publication, and on demand for Python readers (outcome_rows, binding-diagnostic locations) through a new resolved_cleaned_sql.
  • defined_cte_keys skips tokenising text without the keyword with and returns a plain list (every caller treated "unscannable" as empty).
  • SQL-test planning memoises each model query's reserved-CTE check for the run.
  • New E2E test: a cached model's binding diagnostics match the cold compile, before and after a reference placeholder.

Verification

  • Alternating A/B runs on the 10k benchmark project: cold 27.3 → 26.7 s; one-model edit 21.4 → 21.0 s.
  • Identical compiled output and JSON against main for cold and edit compiles on a 1,000-model generated project and a private real project (compile only).
  • compiler-differential with stage captures and coverage gates: 368 projects as expected.
  • cargo test -p sqlbuild-analysis, clippy and fmt (stable); focused pytest on SQL-test planning, cache invalidation, binding catalog, native model analysis and the new test; make test; make check-ci.
  • The new E2E fails when the deferred normalisation is removed.

@kvlonge
kvlonge enabled auto-merge (squash) October 12, 2026 12:55
@kvlonge
kvlonge merged commit 3ffcae2 into main Oct 12, 2026
40 checks passed
@kvlonge
kvlonge deleted the perf/engine-fixes branch October 12, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant