Repository navigation
feat: complete native compiler ownership and CLI replay - #1082
Merged
Merged
Conversation
…elpers WIP for native sqb rename and sqb mv: code-point text edits, the lint interpolation-site scan, MODEL and SCHEMA header edits, and YAML edits over PyYAML-exact node marks composed by sqlbuild-config.
…nder native-preview Ports the model rename and move planner, the column rename walk, model migrate_from, staging and commit with rollback into sqlbuild-refactor, binds them for the Python shell, and runs them behind the new refactoring stage. The refactor integration and e2e tests run under native and native-preview.
Rust planner tests for collisions, migrate_from decisions, rollback and stale files; exact native edit-count integration test; CompiledProject import boundary test; refactor corpus cases with full JSON goldens; stage captures mask the refactor staging folder; fensu-clean layout.
YAML marks follow PyYAML for anchors, aliases, tags and quoted scalars, so anchored declaration files no longer defer. The cross-folder declaration file moves are planned natively over the shared scope placement. Refactoring is always native; the Python planner, staging, commit and their helper tests are removed, with cases ported to Rust.
Property start marks were found by walking back over comment-aware code ends that treat ' #' inside a quoted value as a comment, so a valid tagged source after such a value made rename raise. The composer now records the first property while composing, scanning only between the previous event and the node.
…ranches Every native stage is shipped, so the python engine switch, the NativeStage members it gated and their native-answer counters no longer select anything. SQLBUILD_COMPILER_ENGINE=python and --compiler-engine python now stop with an error that explains the removal. Python branches that only the python engine reached are deleted: JSON SQL-test planning and chain resolution, Python promotion conflicts, the whole-project contract fallback and the attachment else-branches. Native deferrals still hand their inputs to the Python helpers. The compiler differential now compares native with native-preview plus the goldens, and the phase-CPU ratio jobs compare against the base commit.
…ge on native Tests that compared the removed python engine with native now either compare native with native-preview, keeping every explicit expectation, or compare native with the Python fallback it still defers to (project assembly, SQL-test assembly, contracts, semantic completion, type normalisation). Pure parity tests whose Python side was deleted are removed; goldens and the corpus pin that output. The Rust JSON planning entry points become test-only helpers now that their bindings are gone.
…swer counts The shipped stages no longer report native answers, so their <stage>.native entries leave the allow-list. No fallback entry or count changes and every golden is unchanged.
A ref cycle no longer hands the whole project to Python. Like Python's dataflow, the session analyses every model in one unordered wave over the shapes known up front and, like Python's completion pass on a CycleError, keeps those analyses without enrichment or binding re-validation. The cycle flag is part of the analysis-cache key.
…'s Unicode semantics Generated casefold/upper tables per Unicode version, the running Python's text semantics recorded at import, all Polyglot dialects compiled in, arbitrary-size type parameters, stack growth for deep types, and Python's errors for unknown dialects. Deletes the Python type normalization and per-model contract fallbacks.
…ly top-level type fields
…l adapter nullability rules back Python's casefold/upper in CTE facts, legacy analysis and dynamic pivot proofs; pivot parse errors reported as Python reports them; shipped safe_cast rule as data; an adapter's own rules run through an explicit callback whose exceptions reach the caller, bypassing the analysis cache as Python's did.
…red projects Unknown dialect names raise Python's ValueError, projects without an analysis catalog run on a fresh analysis pool, and column references are walked iteratively. Deletes the whole-project Python FAST lineage fallback; the per-model handback for native failures stays.
…atively Project assembly answers per model whether its SQL parses instead of handing the whole project back, and Python's syntax messages come from one native validate/parse_one entry point.
Native model analysis no longer hands models, enrichments, dynamic pivot proofs or whole sessions back to Python. An internal native failure (a panic or a broken invariant) raises NativeCompilerError with its context and fails the compile; adapter rule exceptions and the ValueError Python's analysis raised for a dialect alias keep their original types. Non-ASCII function names now resolve declared return types natively. Deletes the Python whole-project analysis driver, binding waves and dataflow, the SQLite analysis cache, the Python dynamic pivot proof and their fallback-only tests; parity tests now compare uncached, cached and re-proven native runs.
…he wheel A parser panic or unreadable tree in native FAST lineage raises NativeCompilerError instead of re-parsing with the Polyglot wheel, which aborts the process on the same input; invalid SQL keeps its unparsed log. Deletes the per-model Python FAST builder. Analysis-session enrichment no longer maps an analyze_query options rejection to Python's ValueError: compile passes the alias-mapped dialect (postgres -> postgresql), so the rejection is internal.
…puts Native-vs-native cache comparisons prove cache consistency only, so the generated analysis, FAST lineage, type and contract tests now also match digests recorded once from the pre-port Python implementations. Restores the deleted shared-binding tests' explicit diagnostics and lineage expectations as native end-to-end tests, keeps postgres, motherduck and sqlserver alias routes covered at the compile boundary, drops tests and helpers that patched deleted Python seams, and brings the native stubs in line with the raising session API.
…ct and text Ports the semantic completion stage's remaining deferrals: Python's \w, \d, \s, re.IGNORECASE and $ semantics, splitlines boundaries, code-point offsets, casefold and upper beyond ASCII. A missing dialect raises the wheel's TypeError and an unknown one its ValueError, as Python's completion did; an internal failure (a panic, an unreadable tree, a location outside the authored lines, a project without its analysis catalog) raises NativeCompilerError. Deletes Python's recover_output_types, explain_diagnostics, recover_diagnostics, opt-out rejection and metadata checks. The native tests now match outputs recorded from those implementations, including non-ASCII projects and the dialects the full parser build added.
Untyped expression sources take Python's legacy analysis natively, running adapter nullability rules as the session does; a project restored without its catalog infers on a fresh one. Deletes Python's expression-source shape inference and the deferred-shapes fixture.
Models native analysis cannot project (CREATE TABLE AS, INSERT ... SELECT, deeply nested calls) keep the columns Python's parsed-tree inference gave them; table-function bodies, SQL-test fixture interfaces, source rebinding and planner fixture inference run the matching native analysis. A model the session leaves unanalysed is a broken invariant and raises NativeCompilerError. Deletes Python's compact, parsed-tree and CTE fact analyses; the few reference and placeholder helpers left move to reference_names, binding_requests and cte_scopes. The CTE fact parity test matches facts recorded from Python's recovery.
…all natively SQL-test assembly no longer hands tests to Python: non-ASCII contents (Python casefold, \w and re.IGNORECASE keys from generated tables), deep CTE trees, decimal rounding/underflow/overflow under Python's default context, reference and macro scan errors are native. Tests mocking macros keep their model queries from the user's macros, reported as the sql_test_glue.macro_mock_queries extension callback. The macro bridge scans every string natively and reports where Python's scan raises; strings with an unresolvable macro tree or a scan error evaluate their calls in order without the memo. Deletes Python's SQL-test assembly, scope deps and case identity.
Models send name, directory and tags, seeds their tags, and SQL functions their kind, name and tags, so the assembly that computes dependencies can build the project graph.
…rings through native call sites find_macro_call_names, find_nested_macro_call_names and the final macro output check use the native scanner. Strings expanded outside compile inputs evaluate their native call sites in order without the memo, as bridged strings with an unresolvable tree do, so Python's top-level call scanner and its macro_calls.unbridged_expansion fallback are deleted. The bridge requires a supported Python, as discovery already does, instead of falling back to the Python expansion.
…ished analysis session The finished session keeps each model's output columns with their types and nullability, not just names, and each model's dynamic pivot proof from the session's own relation facts, so native consumers read them without re-analysis.
…cture checks Doc comments are single summary lines; entry modules expose one function with helpers, errors and constants in their roles; placeholder defaults live in sqlbuild-sqltext; fallible probes match their errors. Test helpers move out of test modules and carry no conditional control flow, the golden comparison moves into the compiler test helpers, and the analysis-code scan reads the contract codes from the native crate. Two rule exceptions record the process-wide Python text semantics and the nullability callback's manual Debug.
…c proof FinishedSession::model_analysis returns the typed columns compilation publishes for a model the session decides (a proven pivot's fixed columns, else a successful analysis's) and the session's dynamic proof, borrowed from the finished session, for the native compiled-project owner to record without re-analysis.
Each call's arguments are scanned by the native call-site scanner, which now reports where a call's arguments start and the name of a call Python's evaluation fails in, so every call, top level or nested, runs from native bounds. A failing call raises its macro lookup error first, then the scan's message, as Python's evaluation did. Python's macro call scanner, name parser and parenthesis matching are deleted.
Literal types, NULL literals and quoted aliases of a fixture query's first SELECT come from the native parse instead of the wheel's to_dict walk in Python. A golden recorded from Python's _simple_projection_facts covers literals, structured values, quoted non-ASCII aliases, nested and set-operation selects and parse errors.
Python's regex substitute_placeholder_defaults now calls the native placeholder substitution the syntax checks already use, so one implementation serves both.
…ied-reference scans Native analysis sessions share binding results themselves, so Python's binding-share prekeys, shared result keys and the qualified-reference scan they used no longer run anywhere, along with the constants only the deleted Python analyses read.
…ely with the refactor text helpers
kvlonge
enabled auto-merge (squash)
October 11, 2026 15:34
…s and hands off to Python
…ove hits skip Python
kvlonge
disabled auto-merge
October 11, 2026 20:08
kvlonge
changed the title
kvlonge
enabled auto-merge (squash)
October 11, 2026 20:10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Finish native compiler ownership and give unchanged compiles a native CLI path that avoids Python startup.
Changes
Verification
make check-ci;make test: 11,678 passed. Native build, focused tests and strict Clippy passed.