Skip to content

perf: reduce semantic compile analysis cost - #746

Merged
kvlonge merged 1 commit into
mainfrom
perf/chi-301-realistic-compile-performance
Sep 19, 2026
Merged

kvlonge merged 1 commit into
mainfrom
perf/chi-301-realistic-compile-performance

Conversation

@kvlonge

@kvlonge kvlonge commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Why

The existing scale benchmark primarily padded SQL with comments and did not exercise wide declarations, dense tests/audits, or large set-operation ASTs. Profiling a neutral reproducer also found that nested scope and argument materialization caused superlinear time and extreme peak memory during schema-aware analysis.

Changes

  • pin Polyglot's lightweight validation-scope fix from perf: avoid cloning nested validation scopes polyglot#5
  • use targeted native argument access during CTE, filter, and set-operation type analysis
  • add a neutral semantic-density guard with 976 models, 958 tests, 1,645 audits, about 33k declared columns, 5.8 MB model SQL, and 13 MB compiled test SQL
  • protect a 267-branch, 270-column set-operation shape with cold/warm phase budgets
  • retain and document the existing layered, scale, and test-heavy guards separately

Verification

  • uv run pytest tests/unit/src/sqlbuild/compiler/compile/_helpers/test_sql_analysis_columns.py -n auto --dist loadfile (73 passed)
  • semantic-density performance guard (passed, ~45s test runtime)
  • layered performance guard (passed, ~20s test runtime)
  • make test (6,760 passed)
  • uv sync --all-extras && make check-ci
  • public-repository scanner

@kvlonge
kvlonge enabled auto-merge (squash) September 19, 2026 12:49
@kvlonge
kvlonge merged commit 107ab8f into main Sep 19, 2026
18 checks passed
@kvlonge
kvlonge deleted the perf/chi-301-realistic-compile-performance branch September 19, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant