Skip to content

Commit a608f34

Browse files
committed
fix: keep fully cold large compiles within the memory budget
1 parent 3ac87f1 commit a608f34

3 files changed

Lines changed: 6 additions & 4 deletions

File tree

‎docs/dense-compile-performance.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ The guard checks:
2626
| 1,000 | 14s | 1.5GiB |
2727
| 3,000 | 34s | 2.75GiB |
2828
| 5,000 | 55s | 3.25GiB |
29-
| 10,000 | 115s | 4GiB |
29+
| 10,000 | 145s | 4GiB |
3030

3131
These are regression ceilings for the implemented compiler, with headroom for
3232
host variation. They are not a claim of sub-10-second compilation at every size.
@@ -47,8 +47,10 @@ analysis results in batches of 64. Smaller requests retain a single batch. Both
4747
use up to four workers; local resolution views avoid copying CTE definitions
4848
that are already available through the source catalogue.
4949
These limits change scheduling, not analysis or fallback semantics.
50-
Projects with at least 10,000 model projections use two native analysis workers to
50+
Projects with at least 10,000 model projections use one native analysis worker to
5151
leave memory headroom for the isolated custom-rule host after analysis.
52+
The 10k cold ceiling includes headroom for this memory-bounded scheduling; warm
53+
cache hits and small edit batches keep their existing scheduling.
5254

5355
The compiler reuses borrowed syntax-tree facts when it can prove complete reference
5456
binding and infer types without consulting mutable type annotations. Unsupported

‎src/sqlbuild/compiler/compile/_helpers/analysis/compact.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -626,7 +626,7 @@ def _prepare_compact_analysis_batch(
626626
def _run_compact_analysis_batch(*, preparation: CompactBatchPreparation) -> object:
627627
"""Bound resident analysis heaps before the isolated custom-rule host starts."""
628628

629-
workers: int = 2 if len(preparation.projections) >= _LARGE_COMPACT_PROJECT_MODELS else 4
629+
workers: int = 1 if len(preparation.projections) >= _LARGE_COMPACT_PROJECT_MODELS else 4
630630
return orjson.loads(
631631
cast(NativeQueryAnalysisModule, _native).analyze_project_queries_compact_json(
632632
orjson.dumps(

‎tests/e2e/src/sqlbuild/cli/commands/main/compile/test_dense_compile_performance.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@
5959
DenseCompileGuardTestCase(
6060
"dense_models_10000_all_rules",
6161
10000,
62-
115.0,
62+
145.0,
6363
4 * _GIB,
6464
"8125ddde6d3ae172bb15878f95048d66398453c516bbd837b1fd83c04c5982b5",
6565
),

0 commit comments

Comments
 (0)