Repository navigation
refactor: call compile driver and planner stages directly and host CompiledResourceType natively - #1124
Merged
Conversation
…lanner work natively Planning, migration fingerprints, identities, SQL-test planning and table types no longer go through _native module lookups. Tests that replaced those functions now observe native work counters (planner_identity_work_info, migration_fingerprint_work_info), drive planning through project and adapter inputs, or inject table-type drift at the executor hand-off. Freshness errors, cost scopes, progress reporters and the event writer are constructed directly. SQL-test planning and chain requests are Rust structs; their records and pyfunctions are removed.
… identities at discovery ProjectGraph is a native class over the compiled project and its native lineage graph; selection, DAG and Python run-selection readers use it without attribute lookups. Planner scope, loader and static command phases call their native helpers directly. Discovered Python nodes carry their identity, hashed once by the user-code host at discovery; graphs built from hand-made records still hash on demand.
…native views Native models, seeds and functions take adapter target defaults through direct Rust calls instead of Python method dispatch. The project lineage graph reads source, seed and function rows from their native records. Removes NativeAttr ids that no Rust code or pure-Rust crate names.
…directly and count stage work natively Discovery, model configuration and the lane A compile driver files (assembly*, project, discovery, inputs, config_validation) now call the native stage functions directly instead of looking them up on the sqlbuild._native module. Tests that spied on those module attributes read compile_stage_work_info counters (SQL test assembly, pivot proofs, declaration scopes, promotion conflicts, lineage graph builds) or the stage's own answers instead. Removed: the externally_described_models pyfunction (no Python caller), 30 unused NativeAttr ids, the header worker-pool error test and the unretained-model-facts test (both needed a replaced module function to reach an internal invariant).
…ively during analysis setup assembly_analysis builds the declared relations, known function names and known declared types with direct Rust calls. The declared_relations, known_function_names and known_declared_types pyfunctions had no Python callers and are removed with their stubs and NativeAttr ids.
… completion as direct calls Assembly calls the three analysis stages in Rust. Stage tests take the stage inputs from the test-support assembly_stage_inputs(stage, ...), which assembles like assemble_project and stops before the stage, and read stage counts from compile_stage_work_info (including expression shape batches). Removes the invariant-only test that replaced model analysis with an empty result.
…ry directly from commands compile_driver analysis, pipeline, command and phases call lineage, graph selection, unit-test selector checks, adapter-name resolution, contracts, rules, connection config, manifest and reuse-store stages in Rust, and every command calls discover_project_inputs / build_compiled_project directly. Tests that replaced those module attributes now use stage counters (project discoveries, lineage, contracts), a failing provider module, adapter hooks on a manual clock, or a PreparedCompileArtifacts subclass; the compile case of the injected rich-lineage panic test is dropped because compile no longer reads the module attribute.
CompiledResourceType is registered with the native string enums in sqlbuild._native (same members, values, str, repr and StrEnum behaviour) instead of being defined in sqlbuild.compiler.compile.types. Rust reads it through the record registry rather than importing the Python module or the facade host entry; Python importers take it from sqlbuild._native. Stage captures keep its previous qualified name through the hosted shape table, so captures are unchanged.
…d merge lane updates
… make the compile interrupt test deterministic
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The compile driver, planner and discovery stages still reached SQLBuild's own native functions through Python module lookups. CompiledResourceType was still a Python enum that Rust imported. An interrupt during compile could also be lost while freed memory was being released after rules.
Changes
Verification