Skip to content

perf: integrate conservative cold compiler optimizations - #776

Merged
kvlonge merged 6 commits into
mainfrom
perf/cold-compiler-production-integration
Sep 22, 2026
Merged

kvlonge merged 6 commits into
mainfrom
perf/cold-compiler-production-integration

Conversation

@kvlonge

@kvlonge kvlonge commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Why

Reduce cold compiler overhead while preserving normal validation, isolated custom rules, cache identities and artifact publication gates.

Changes

  • Reuse conservative borrowed-tree binding and annotation-independent type evidence; retain normal validation for unsupported shapes and warning-producing joins.
  • Stage bounded cold-project artifacts alongside selected rules, then publish at the normal successful write phase; preserve unchanged files and stale-file cleanup.
  • Use one native analysis worker for SQL-heavy batches (10k+ projections, 48MiB+ prepared SQL) to bound compiler/custom-host memory; retain parallel scheduling for lighter batches.
  • Avoid eager nullability fallback, unnecessary output chunking and unused custom-rule fingerprints when the rules cache is disabled.
  • Add native differential and real CLI/DuckDB coverage for diagnostics, cache modes and publication failures.

Verification

  • 163 focused Python tests passed; native query-analysis tests passed.
  • uv sync --all-extras and make check-ci passed.
  • The full cold 10k guard passed locally inside a 4GiB cgroup with zero cache hits and its expected fingerprint (107.09s). Its ceiling is 145s for memory-bounded scheduling; CI remains the final gate.
  • Independent read-only review and bounded warning-preservation follow-up completed.

@kvlonge
kvlonge enabled auto-merge (squash) September 22, 2026 15:56
@kvlonge
kvlonge merged commit 0356503 into main Sep 22, 2026
29 checks passed
@kvlonge
kvlonge deleted the perf/cold-compiler-production-integration branch September 22, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant