Skip to content

refactor: port compiler stages, result output and command dispatch to the native extension - #1100

Merged
kvlonge merged 31 commits into
mainfrom
feat/native-group-1
Oct 13, 2026
Merged

kvlonge merged 31 commits into
mainfrom
feat/native-group-1

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Why

Rust-hosted SQLBuild port (CHI-585): five worker batches combined into one PR. Phase 1 (CHI-589) moves more compiler stages native; Phase 3 (CHI-592) moves execution-result output and command dispatch.

Changes

  • Compiler: native BindingCatalog and schema validation, graph, references, fingerprint read/write, contracts, contract adoption, authored values, rename/mv string search.
  • Discovery: Python function/hook/module listing, UDF static reading, schema columns, directory snapshot, D016; offline scope index load, cache and build.
  • SQL test assembly, auditing (identities, measurements, audit SQL), storage policies, cursor roles, input reads, syntax checks, pivot requests, reference scanning (P012).
  • CLI: execution-result JSON for every run command and the terminal-event index; command dispatch through a native route table, with error projection and exit codes in Rust (run_cli).
  • runtime compute logs, event export and output capture marked to stay Python.
  • Movable Python: 115,330 → 103,112 lines.

Intentional differences: expected errors in --debug logs carry different traceback frames; several internal Python APIs are removed or renamed; the scope-index cache misses once after upgrade.

Verification

  • Workers compared old vs new: 363–372 compile corpus projects (output and target/), real CLI runs on 17 DuckDB projects for result documents, 53 dispatch scenarios × 2 launchers: no differences.
  • Compiler differential with full coverage: 371 projects; byte-identical compile output on a 1k-model project and a large private project.
  • 10,540 focused tests incl. scope, SQL-test, audit, check, plan and compile-reuse e2e; make check-ci; hygiene scan.
  • Independent review of the combined diff: no findings.

…n native

BindingCatalog is now a native class; schema validation batches, the quoted
identifier policy and the dialect constants run in Rust. Delete the Python
catalog, schema-validation, identifier-case and test-only normalization
wrappers.
…ively

Closure, inversion and path-node walks, attached-audit and hook-read edges,
hook names, SQL ref keys and the native graph's dict views are now native
functions. Delete the Python graph helpers and entry modules; the package
keeps only its edge and view models.
…rker checks natively

Relation-name extraction, project-relation matching, runtime relation indexes,
source relation rendering, unresolved-marker assertions and the P008 remedy
and old-name messages are native functions. Delete the Python wrappers; the
package keeps its models, kinds and the regex patterns the planner reads.
…ables natively

Query and function-definition hashing, fingerprint DDL and latest-read SQL,
latest reads and the retrying single and batched writes run natively through
the adapter's execute callable. Delete the Python helpers and entry modules;
the package keeps its row models, vocabulary and exceptions.
…atively

Contract evaluation and K011 promotion conflicts now return compile
diagnostics straight from native code, including the output-column location
recovery. Delete the Python wrappers; compile reaches both through
sqlbuild._native.
Physical-versus-authored comparison, MODEL and source declaration edits and
the remaining-findings rule run natively. Delete the Python helpers and the
compare entry; the write orchestration that recompiles the project stays in
Python.
… vars natively

Optional named string and boolean validation and project var text rendering
are native functions. Delete the authored_values package, including the
change-policy helpers native config already replaced and the field
validators only tests used.
…ively

The project Python file walk and string-location search for sqb rename and
sqb mv run natively. Delete the Python helper and entry module.
The native assembly now returns NativeSqlTestAssembly objects with their
compiled tests, keyed P013 diagnostics and failures. Delete the Python
assembly helpers and the artifact-planning pass-through; the compile target
writer calls the planner's artifact planner directly.
…SQL natively

Audit gate identity hashing, measurement evaluation, audit SQL reference
rendering and schema audit list parsing run natively. The identity hashing
reproduces json.dumps exactly and lives in sqlbuild-analysis with tests
pinned to the Python hashes. Delete the Python helpers and entry modules;
the public threshold builders and audit factory entry points stay.
…eads natively

Inherited time-travel retention and table types are resolved inside the
native config session, cursor input roles resolve natively, and the
compile input-read recorder is a native class. Delete the Python modules;
table type tests drive the native config builder.
…uests natively

Model placeholders, the model and hook SQL assembly validates, and the
standalone pivot proof requests are built natively. Delete the Python
analysis helpers.
SQL reference scanning and extraction, with P012 reporting for rejected
calls, run natively. Delete the Python reference helpers.
@kvlonge
kvlonge enabled auto-merge (squash) October 13, 2026 09:27
@kvlonge
kvlonge merged commit a3c431c into main Oct 13, 2026
40 checks passed
@kvlonge
kvlonge deleted the feat/native-group-1 branch October 13, 2026 14:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant