Skip to content

refactor: construct native records and call native functions directly across the bindings - #1125

Merged
kvlonge merged 28 commits into
mainfrom
feat/chi599-batch-9
Oct 17, 2026
Merged

kvlonge merged 28 commits into
mainfrom
feat/chi599-batch-9

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 17, 2026

Copy link
Copy Markdown
Contributor

Why

Across the bindings, Rust still built SQLBuild's own native records through module lookups, called its own native functions through sqlbuild._native, and copied records through dataclasses.replace.

Changes

  • Record classes, string enums and enum members are captured when the module initialises and built directly. The record-class and member registry lookups are removed.
  • About 300 native functions and pyclasses are called or constructed directly, including compile diagnostics, macro and context store tokens, the rules request, schema revalidation, source reanalysis, the dbt pipeline and the progress reporters.
  • Record copies use a native rebuild instead of dataclasses.replace. Chained and dynamic field reads on our own records use direct native field reads.
  • Tests that replaced native functions now check observable behaviour or native stage counters. One seam-only test was removed.

Verification

  • make test and make check-ci passed, with all optional dependencies available.
  • DuckDB E2E command suites passed.
  • Compiler differential: 371 projects, goldens unchanged. Planner differential: 11 projects matched a fresh main baseline.
  • The Windows cross-check passed.

…scenario, seed and test commands

Command bodies in plan_command, query_command, scenario_command, seed_command
and sql_test_command (plus the rules custom evaluation and offline refactor
compile) now call the Rust functions and construct the progress reporter
pyclasses directly instead of looking them up on sqlbuild._native.

The scenario command presentation tests now compile a real one-scenario
DuckDB project instead of replacing native compile seams.
Add records::construct (record_class, new_record, new_record_with, enum_member, enum_from_value) holding every record class and string enum captured once at init, and use it for analysis bindings instead of a module attribute lookup per construction.
…les bindings

The host payload facts test now asserts the observed dropped fields instead of
spying on the CompiledModel constructor and dataclasses.replace seams.
…nd semantic bindings

The native analysis seam e2e test no longer spies on
infer_native_expression_source_shapes; the stage-work counter it already
checks covers that stage.
…helpers directly

Semantic completion, source rebinding and native rules evaluation now call the
Rust functions instead of sqlbuild._native attributes. Tests that replaced
those attributes now read new stage-work counters (schema_revalidations,
source_rebinding_analyses, rules_memo_reuses) or build the rules request
directly; the request's facts JSON now includes its scope index.
…irect calls

Scope, SQL reference and macro expansion bindings call report_compile_diagnostic,
macro_store_token and context_store_token directly instead of carrying the
sqlbuild._native callables in their class tables.
…ses.replace in m–z bindings

Chained and dynamic getattr reads in the macro_bridge … workspace_commands
bindings go through records::fields::field, and dataclasses.replace calls go
through records::fields::replaced / replaced_keywords, which rebuild a native
record from its own values.
Planner record helpers construct through records::construct, the scope command
loads its default index with the Rust loader, and the now-unused NativeItem /
registry::item lookup is removed.
…runtime bindings

Replace per-use sqlbuild._native class lookups with records::construct helpers (record construction, enum member and value lookup).
…ompute_logs bindings

Replace per-use sqlbuild._native class lookups with records::construct helpers (record construction, enum member and value lookup).
…sor bindings

Replace per-use sqlbuild._native class lookups with records::construct helpers (record construction, enum member and value lookup).
…ry bindings

Replace per-use sqlbuild._native class lookups with records::construct helpers (record construction, enum member and value lookup).
… bindings

Replace per-use sqlbuild._native class lookups with records::construct helpers (record construction, enum member and value lookup).
Raise visibility of pyfunctions in plan, refactor, result, rules and workspace bindings that command bindings now call directly.
…nd_runtime bindings

Replace keyword calls through sqlbuild._native attributes with direct Rust calls to the pyfunctions. Entry points that tests replace on the module are left for a follow-up test rewrite.
…t_report bindings

Replace keyword calls through sqlbuild._native attributes with direct Rust calls to the pyfunctions. Entry points that tests replace on the module are left for a follow-up test rewrite.
…ommand bindings

Replace keyword calls through sqlbuild._native attributes with direct Rust calls to the pyfunctions. Entry points that tests replace on the module are left for a follow-up test rewrite.
… bindings

Replace keyword calls through sqlbuild._native attributes with direct Rust calls to the pyfunctions. Entry points that tests replace on the module are left for a follow-up test rewrite.
Call dbt init, debug and execution, profile, connection, adapter, project graph, declaration scope and fixture inference entry points directly from Rust. Tests now exercise real behaviour: dbt workspaces and profiles on disk, a stand-in dbt executable, a SQLBuild project with a real connection, and a new native fixture_inference_count work counter. The declaration-scope syntax test only checked the arguments passed to the replaced entry point and is removed; the formatting test above it still covers caller-held syntax.
…ectly in a-l bindings

Construct native pyclasses (progress reporters, styles, event writers, runners, cost scopes, validators, loops) in Rust, call the remaining keyword, positional and value-held entry points directly, dispatch CLI command implementations through a typed table, and hold launch, progress and warehouse-scope context variables natively. The format-selection work test now reads native stage counters (format_lint_batches, format_lint_files, build_project_graph) instead of replacing entry points.
…ively in a-l bindings

Route non-literal attribute reads in a-l bindings through records::fields::field and replace dataclasses.replace with records::fields::replaced / replaced_with (new: takes a keyword dictionary). Keep the compiled code identity lookup on the extension module, which tests replace to simulate a rebuilt installation.
The rich-lineage panic e2e test replaces build_project_column_lineage in the CLI process to prove native panics fail the command; no native trigger exists, so the call stays a module lookup with positional arguments.
…ds in a-l bindings

Convert chained attribute reads on own records (source loop contexts, schema columns, macro enforcement, settings, clone policy, record fields) to records::fields::field, and rebuild compiled-project sources with replaced_with instead of dataclasses.replace.
@kvlonge
kvlonge enabled auto-merge (squash) October 17, 2026 04:48
@kvlonge
kvlonge merged commit 9376f69 into main Oct 17, 2026
40 checks passed
@kvlonge
kvlonge deleted the feat/chi599-batch-9 branch October 17, 2026 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant