Repository navigation
feat(native): discover declarations, SQL tests and YAML files natively - #999
Merged
Merged
Conversation
With the native engine, Rust walks the canonical, global and named declaration roots on the pass's shared tree, validates the declaration group, scoped root and named role layout, and returns the macro, enum and constant file facts and the declaration groups. Python records them only for a valid layout and rescans an invalid one to raise its own error. Reading and parsing declaration files stays in Python.
With the native engine, Rust selects, reads and splits SQL test files into TEST blocks, parses each header and rejects unsupported keys, and cleans bodies like inspect.cleandoc; scenario files get the same header handling. Python keeps the block and scenario semantics, including parameter value normalisation, so the first failure is unchanged.
With the native engine, Rust reads the source, model schema and seed declaration files Python selects and loads each with the PyYAML-compatible loader. A file the native loader cannot reproduce is loaded by Python, and an unreadable file is re-read by Python for its error. Validation of the loaded documents stays in Python.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Milestone M2 moves project discovery to Rust behind the compiler engine switch. After model files, discovery time with
SQLBUILD_COMPILER_ENGINE=nativewas spent on the declaration layout, SQL test files and YAML loading. This change makes those native. Output stays byte-identical to the Python engine, and the default engine stayspython.Changes
TESTblocks, parses headers, rejects unsupported keys and cleans bodies (aninspect.cleandocport). Python keeps block and scenario semantics, including parameter normalisation, so the first error is unchanged.Verification
make compiler-differentialwith native on Python 3.12: 114 projects identical (8 new layout failure cases); dense 3k identical. Dense 3k discovery: ~3.7 s (python) → ~1.5 s (native).cleandocoracle.cargo test --workspace --all-features, clippy/fmt 1.98.0,make check-native-layering,make check-ci,make test.