Repository navigation
refactor: port clone, load, check, test, seed and diff commands, rule caches and state helpers to the native extension - #1102
Merged
Merged
Conversation
…ersions natively The new plain-Rust sqlbuild-state crate computes the content-addressed migration, column-rename and old-name view event IDs, renders the idempotent event lookup and typed INSERT statements, and normalizes, hashes, compares and age-checks source freshness data versions. Stored IDs and hashes are pinned to values the Python implementation wrote. Reads and writes still go through the adapter, and warehouse orchestration stays in Python.
Secret redaction of diagnostic text and structured logging fields runs in sqlbuild-core with Rust regex. The Python logging handlers, formatters, filters and the LogRecord field collection stay in Python and are listed as stays in python-boundary.toml.
SQL rule cache keys (lint settings, rule codes, model SQL and dynamic column contract, non-model file text), stored-entry lookup and the cache file now live in sqlbuild-rules as pure functions with a new key format (sql-rules-v6), so the first run after upgrading misses once. A new integration test pins cold misses, warm hits and invalidation on model, threshold, exception-policy and rule-selection edits.
…natively Custom-rule identities (code identity, host runtime, rule metadata, source fingerprint, rules settings, dialect, project root), subject identities, read-set lookup, result merging, finding validation and ordering, and the cache file now live in sqlbuild-rules as pure functions behind a CustomRuleCache binding, with a new key format (custom-rules-native-v1), so the first run after upgrading misses once. Fact digests stay with the Python fact views the rule hosts read. A new integration test pins cold misses, warm hits, a hit through an unread model edit, and invalidation on a rule source edit.
The native loader now returns the RulesConfig models directly and raises RulesError or RulesConfigError itself; the allow_exceptions policy check moves to sqlbuild-rules. The JSON configuration hand-off and its Python normalisation are removed. Selector resolution against the compiled project stays with the planner scope in Python.
The rules request binding now reads compiled SQL tests, scenarios and initial findings straight from the Python objects, and evaluation, finalisation, the SQL rule cache lookup and the custom-rule cache return Finding models built natively. Finalisation keeps each original finding's fixable mark natively, and undecodable cached SQL findings are dropped as misses inside the lookup, so its handled-error entry is removed. The Python row builders and finding decoders are deleted; a new NativeRulesRequest.sql_facts_json accessor lets tests inspect the facts sent to the engine.
…dules Thread pools, futures and stop events around a rules run move to run/rule_threads.py, and custom-rule submission beside native evaluation moves to engine/custom_dispatch.py. Run planning, SQL rule evaluation and result finalisation stay in run/rules.py as plain functions. The two new modules and engine/custom_rules.py, which dispatches to the rule-host worker pool, are listed as user-code stays.
Built-in Rule models, their declared options and guidance are built by a native builtin_rules binding from the sqlbuild-rules catalogue, and the early-lint SQL code selection runs natively. The JSON catalogue hand-off and the Python option table are removed. Loading custom rule modules and validating their declared options stay in Python with the user code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Rust-hosted SQLBuild port (CHI-585), third combined PR: more Phase 3 (CHI-592) commands, the Phase 1 (CHI-589) rule engine split, Phase 4 (CHI-593) state helpers, and the CHI-599 guard for Rust-to-Python calls.
Changes
sqb clone,load,check,test,seed,diff; dbt interop static parts (profiles,dbt init, plan, manifest);sqb dbt ... --jsonprogress now goes to stderr (pre-existing bug).sqlbuild-statecrate for migration event IDs/SQL and freshness versions.tests/unit/scripts/native_module_paths.Intentional differences: cache formats; diff cells no longer get Rich auto-highlighting/markup; traceback frames; internal Python APIs removed or renamed.
Verification
make check-ci.