Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
8257340
feat(native): answer legacy analysis natively behind the preview engine
kvlonge Oct 9, 2026
a0e18a1
test(native): cover real adapter rules and non-ASCII names in the ana…
kvlonge Oct 9, 2026
c415e44
test(native): set the seeded analysis parity floors to their exact co…
kvlonge Oct 9, 2026
67ad855
fix(native): reconcile legacy analysis with shared analyses and sessi…
kvlonge Oct 10, 2026
344c475
feat(native): make native analysis the default compiler path
kvlonge Oct 10, 2026
eb9eb2e
test(native): record default-engine analysis in the allow-list and pe…
kvlonge Oct 10, 2026
ab46937
perf(native): skip the set-operation regex when no keyword can match
kvlonge Oct 10, 2026
d4322f9
feat(native): cache native model analyses in the shared native store
kvlonge Oct 10, 2026
0e85cc1
perf(native): send each source and SQL test file's text to metadata c…
kvlonge Oct 10, 2026
17dfc1b
test(native): prove native analysis cache invalidation and uncached e…
kvlonge Oct 10, 2026
4d6a9df
perf(native): reuse exact shared analyses across native analysis batches
kvlonge Oct 10, 2026
af9ae6d
ci: gate the native engine's phase CPU against python
kvlonge Oct 10, 2026
cf13053
perf(native): analyse large waves in chunks and key shared results by…
kvlonge Oct 10, 2026
1c4cf3e
docs: describe shipped native analysis and its model analysis cache
kvlonge Oct 10, 2026
d2177ea
test(native): regenerate the native allow-list after the M3 close-out
kvlonge Oct 10, 2026
0f53ac9
test(native): pin Python-internals tests to the Python engine
kvlonge Oct 10, 2026
c06136c
refactor(native): meet the static checks for the native analysis cache
kvlonge Oct 10, 2026
6820135
test(native): guard the text the dense semantic metadata request carries
kvlonge Oct 10, 2026
96a2ec5
test(native): align cache and memory guards with native analysis
kvlonge Oct 10, 2026
16407c0
test(native): count the extra cache priming and uncached head compile…
kvlonge Oct 10, 2026
f7b89a5
fix(native): store natively answered legacy and failed model analyses
kvlonge Oct 10, 2026
4006e0e
perf(native): chain relation-table digests instead of rehashing every…
kvlonge Oct 10, 2026
e0e1d7f
test(native): give the 3k column lineage inspections wall headroom on…
kvlonge Oct 10, 2026
1d783dd
test(native): scope temporary memory allowances for the compiler conv…
kvlonge Oct 10, 2026
417f9d0
test(native): scope approved temporary edit performance allowances
kvlonge Oct 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 14 additions & 8 deletions .github/workflows/verify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ jobs:
run: >-
make compiler-differential-shipped
COMPILER_DIFFERENTIAL_ARGS="--stage-captures --require-discovery-coverage
--require-render-coverage
--require-render-coverage --require-analysis-coverage
--evidence-dir ${{ runner.temp }}/compiler-differential-evidence/shipped"
- name: Upload differing projects' targets, stderr and compiled-test stats
if: failure()
Expand Down Expand Up @@ -606,25 +606,27 @@ jobs:
include:
- kind: dense
max_ratio: "1.10"
edit_max_ratio: "1.20"
edit_max_cpu_ratio: "1.75"
- kind: fresh
max_ratio: "1.10"
- kind: dense
label: native-preview contracts vs python
label: native contracts vs python
base_engine: python
head_engine: native-preview
head_engine: native
gate_phase: contracts_cpu_ms
max_ratio: "1.00"
- kind: dense
label: native-preview model analysis vs python
label: native model analysis vs python
base_engine: python
head_engine: native-preview
head_engine: native
gate_phase: model_analysis_cpu_ms
compile_args: --compile-arg=--select=*
max_ratio: "1.50"
- kind: dense
label: native-preview custom rules vs python
label: native custom rules vs python
base_engine: python
head_engine: native-preview
head_engine: native
gate_phase: custom_rules_cpu_ms
max_ratio: "1.25"
steps:
Expand Down Expand Up @@ -653,7 +655,7 @@ jobs:
- name: Install base dependencies
working-directory: base
run: uv sync --locked --all-groups --all-extras
- name: Compare python and native-preview phase CPU on head
- name: Compare python and native phase CPU on head
if: matrix.gate_phase
env:
KIND: ${{ matrix.kind }}
Expand All @@ -679,10 +681,14 @@ jobs:
env:
KIND: ${{ matrix.kind }}
MAX_RATIO: ${{ matrix.max_ratio }}
EDIT_MAX_RATIO: ${{ matrix.edit_max_ratio }}
EDIT_MAX_CPU_RATIO: ${{ matrix.edit_max_cpu_ratio }}
run: |
uv run python -m scripts.compare_compile_performance \
--kind "${KIND}" \
--modes cold warm edit \
${EDIT_MAX_RATIO:+--edit-max-ratio "$EDIT_MAX_RATIO"} \
${EDIT_MAX_CPU_RATIO:+--edit-max-cpu-ratio "$EDIT_MAX_CPU_RATIO"} \
--max-ratio "${MAX_RATIO}" \
--models 3000 \
--base-root base \
Expand Down
10 changes: 10 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,16 @@
- Performance fixes must not reduce coverage. Show that diagnostics are unchanged or a strict,
validated superset on the dense benchmark and a real-project run.
- Do not raise performance budgets or tolerances to make a regression pass.
- Temporary user-approved exception (2026-10-10), removable after post-conversion memory
optimisation: only the 10,000-model case in `test_fresh_process_compile_cache_performance.py`
may use 3 GiB peak RSS instead of 2.5 GiB; release-performance `lineage column trace` may use
a 2.0 peak RSS ratio and `dag --json` a 1.5 ratio instead of 1.25. Smaller cache cases,
cold-only guards, all other commands, and all speed/correctness limits retain their budgets.
The prohibition above applies to every other budget or tolerance increase.
- Additional temporary user-approved exception (2026-10-10): the dense 3,000-model
base/head edit comparison may use 1.20x wall and 1.75x CPU; release-performance
`compile (one-model edit)` may use 1.35x peak RSS. Remove after post-conversion
optimisation. Cold/warm modes, other workloads and correctness gates remain unchanged.

## Native SQL (Polyglot) Changes

Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,7 @@ compiler-differential:
$(COMPILER_DIFFERENTIAL_GATES) $(COMPILER_DIFFERENTIAL_ARGS)

# python vs native-preview above covers every native stage; this keeps the shipped `native` default
# (native discovery and rendering) and its goldens covered on the same corpus.
# (native discovery, rendering and analysis) and its goldens covered on the same corpus.
compiler-differential-shipped:
env PYTHONUNBUFFERED=1 uv run python -m scripts.run_compiler_differential \
--engines python native --corpus fixtures examples seeds failures \
Expand All @@ -319,7 +319,7 @@ compiler-baselines:
$(MAKE) compiler-differential COMPILER_DIFFERENTIAL_ARGS="$(COMPILER_DIFFERENTIAL_CI_ARGS) \
--require-analysis-coverage --native-fallbacks update --goldens update"
$(MAKE) compiler-differential-shipped COMPILER_DIFFERENTIAL_ARGS="$(COMPILER_DIFFERENTIAL_CI_ARGS) \
--native-fallbacks update --goldens update"
--require-analysis-coverage --native-fallbacks update --goldens update"

compiler-differential-dense:
env PYTHONUNBUFFERED=1 uv run python -m scripts.run_compiler_differential \
Expand Down
7 changes: 4 additions & 3 deletions crates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,10 @@ stages run natively:
| `native` | The default: native stages that passed their flip gate (`shipped` tier). |
| `native-preview` | Opt-in: shipped stages plus stages still in development (`preview` tier). |

The shipped tier covers discovery and rendering: declaration files and scopes, model config,
reference extraction, the model loop, macro calls and the macro-call store, and attachments. SQL
analysis, contracts, lineage, SQL-test glue and project assembly are still `preview`. A failing
The shipped tier covers discovery, rendering and analysis: declaration files and scopes, model
config, reference extraction, the model loop, macro calls and the macro-call store, attachments,
the type system, model analysis and its `model-analyses.bin` cache, semantic checks, contracts,
lineage facts, SQL-test glue and project assembly. A failing
render raises its first error directly, as the Python stage would, and runs each macro call at
most once; a divergence the bridge detects is reported as a native stage mismatch. Because the
macro-call store is shipped, the
Expand Down
1 change: 1 addition & 0 deletions crates/sqlbuild-analysis/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ regex.workspace = true
serde.workspace = true
serde_json.workspace = true
sha2.workspace = true
sqlbuild-cache.workspace = true
sqlbuild-core.workspace = true
sqlbuild-model-config.workspace = true
sqlbuild-scopes.workspace = true
Expand Down
Loading
Loading