Skip to content

feat(native): enrich analysis inputs natively behind the preview engine - #1055

Merged
kvlonge merged 3 commits into
mainfrom
feat/native-model-analysis-enrichment
Oct 9, 2026
Merged

kvlonge merged 3 commits into
mainfrom
feat/native-model-analysis-enrichment

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Why

Preview still called the wheel for input enrichment and pivot proofs; both move native.

Changes

  • Enrichment runs natively via analyze_query with Python's projection and lineage. It defers wherever Python leaves its fast path: CTE fact recovery, NULL filters, normalisation or analysis failure, non-select shapes, casts with no upstream column, non-ASCII function names.
  • Pivot proofs run natively, in the session and via the seam for unanalysed models. Parse errors and non-ASCII names defer (dynamic_pivot).
  • Rust pivot cases; three failure-corpus cases. Wheel-site report test (rebased on feat(native): run semantic checks natively behind the preview engine #1050): preview expects only metadata_checks deferrals, as input_enrichment no longer occurs.

Test expectations added (parity test):

  • Two contract-enforced CTE models where Python's CTE recovery changes types; without the deferral check they mismatch.
  • input_enrichment deferrals <= 120 (now 104): every-enrichment-defers fails.
  • Native enrichments >= 90 (now 106), counted at native lineage, so fallbacks do not count.
  • Pivot proof minimums for the new generated shapes.
  • New: a pivot outside --select analysis is proven natively, with no wheel call.

Sabotage, run and reverted: deleting the CTE check gives 3 mismatches; all native enrichments deferring gives 495 deferrals; a monkeypatched native session gives 0 native enrichments; a monkeypatched native pivot proof hits the wheel.

Verification

  • cargo test --workspace --all-features, clippy, fmt (1.99.0) before the rebase.
  • Before the rebase, make compiler-differential (CI flags): 302 identical, enrichment and dynamic_pivot wheel sites 0, 12 legacy_analysis deferrals. -shipped: 293 identical. tests/e2e/scripts/compiler_differential passes.
  • make check-ci, make test, hygiene.

Residuals:

  • No corpus project hits the CTE-recovery deferral; generated parity projects do.
  • Pivots fall back to the wheel when the whole session hands back.
  • Any non-ASCII name defers every pivot proof.

@kvlonge
kvlonge enabled auto-merge (squash) October 9, 2026 04:38
The native model analysis session now re-analyses models with their
known input shapes itself, deferring to Python only where Python's CTE
fact recovery reads the parsed query. It also proves dynamic pivot
contracts natively, inside the session and for models it does not
analyse, deferring on parse errors and non-ASCII names.
@kvlonge
kvlonge force-pushed the feat/native-model-analysis-enrichment branch from ad9a6d6 to a86b008 Compare October 9, 2026 06:05
…lysis-enrichment

# Conflicts:
#	tests/e2e/scripts/compiler_differential/test_compiler_differential.py
@kvlonge
kvlonge merged commit 762eac9 into main Oct 9, 2026
37 checks passed
@kvlonge
kvlonge deleted the feat/native-model-analysis-enrichment branch October 9, 2026 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant