Skip to content

feat(native): resolve declaration scopes natively behind a native-preview engine tier - #1006

Merged
kvlonge merged 7 commits into
mainfrom
feat/native-scopes
Oct 7, 2026
Merged

kvlonge merged 7 commits into
mainfrom
feat/native-scopes

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Why

The compiler moves to Rust stage by stage. Declaration scopes are next. A stage should only become the shipped default after it passes its own flip gate, so stages still in development need a separate preview engine.

Changes

  • Engine tiers: python (oracle), native (default; shipped stages: discovery) and opt-in native-preview (adds stages in development). Each stage declares its tier once in NATIVE_STAGE_TIERS. Preview keeps its own compiler, Rules and reuse stores.
  • New sqlbuild-scopes crate: the scope index with Python's record orders and diagnostics, lookup orders, visibility, relationship grants, and a dialect-aware scan of SQL tests and scenarios for expected models. Anything Python would reject, or a non-ASCII character at a code position, defers to Python. Under native-preview, build_declaration_scope and sqb scope use it. Error messages stay in Python.
  • PyO3 bindings move into per-domain folders (analysis, boundary, discovery, rules, scopes, sqltext) in a separate commit with no body changes.
  • The differential compares python with native-preview on the full corpus. A new compiler-differential-shipped step compares python with native on seeds and failures; the 3.13/3.14 job compares python with native.

Verification

  • cargo test, clippy and fmt (1.98.0), make check-native-layering, make check-ci, make test.
  • Engine-parity tests: crafted and 120 seeded scoped projects. Expected-model scan: 15,000 generated bodies across five dialects (match or defer).
  • Tier tests: stage gating, separate stores and reuse, and the harness under both engines. E2E compile, lineage and scope CLI tests pass locally (performance budgets excepted on a loaded machine).
  • make compiler-differential with render coverage: 144 identical. Shipped: 138 identical. Dense 3k: identical.

The new sqlbuild-scopes crate projects the scope builder's walked
resources and declarations into the static scope index with the Python
builder's record orders, duplicate and naming diagnostics and macro
dependency usages, the lookup's canonical orders and repr-ordered
groups, declaration visibility and the expected-model and tested-macro
relationship grants. Any path Python would reject defers the stage.
… engine switch

With the native engine, the scope index, its validation, relationship
grants and the scope lookup are built by sqlbuild-scopes and
materialised as the same Python records, orders and mappings. Python
keeps the relationship-name extraction and every error message: an
invalid index or a relationship fault raises exactly as before, and a
project the native index cannot reproduce runs the Python stage. sqb
scope uses the same path. The Python engine is unchanged.
The failure corpus gains scope cases: a duplicate scoped declaration,
inaccessible local and scoped declarations, the first of two visibility
errors, a test outside its relationship grants and a malformed expected
model. A harness test compares a scoped project with expected-model and
macro-test grants under both engines.
Move the PyO3 binding helpers into per-domain folders (analysis, boundary, discovery, rules, scopes, sqltext) and split the flat function module along the same lines. Function bodies are unchanged; only module paths and registration move.
Each native stage declares a shipped or preview tier in one registry. The native engine runs shipped stages only (discovery); native-preview adds stages still in development (declaration scopes). Preview keeps its own compiler, Rules and reuse stores. The per-PR differential compares python with native-preview on the full corpus and python with native on the seeds and failure corpus.
Under native-preview the declaration scope reads each model-mode SQL test's and scenario's __expected__<model> names with a native port of the Python top-level CTE scan, using the adapter's lexical rules. Any text Python would reject, or a non-ASCII character at a code position, defers that body to the Python extraction, so names and fault messages stay byte-identical.
@kvlonge
kvlonge enabled auto-merge (squash) October 7, 2026 13:43
@kvlonge
kvlonge merged commit 799db35 into main Oct 7, 2026
36 checks passed
@kvlonge
kvlonge deleted the feat/native-scopes branch October 7, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant