Skip to content

Commit d4c7157

Browse files
authored
feat(native): discover models natively behind the compiler engine switch (#995)
* feat(native): discover SQL model files natively behind the compiler engine switch Add the sqlbuild-discovery crate with a parallel directory walk that keeps Python's glob and sort semantics, and native reading and parsing of SQL model files with Python's exact discovery messages, locations and ordering. With SQLBUILD_COMPILER_ENGINE=native the discovery stage takes its model files from the native walk; every other collection still runs in Python. Native discovery skips the Python discovery fact cache. * fix(native): defer model discovery to Python when it cannot match exactly Run the Python path when the running Python's Unicode database differs from the native character table or the project root is not UTF-8, and share the native models walk with the pass's directory snapshot. * test(compiler): pass every frontier stage through the perturbation hooks
1 parent 3f13bc3 commit d4c7157

90 files changed

Lines changed: 3927 additions & 77 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Cargo.lock‎

Lines changed: 11 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ members = [
33
"crates/sqlbuild-core",
44
"crates/sqlbuild-sqltext",
55
"crates/sqlbuild-config",
6+
"crates/sqlbuild-discovery",
67
"crates/sqlbuild-analysis",
78
"crates/sqlbuild-rules",
89
"crates/sqlbuild-python",
@@ -17,6 +18,7 @@ order = [
1718
"sqlbuild-core",
1819
"sqlbuild-sqltext",
1920
"sqlbuild-config",
21+
"sqlbuild-discovery",
2022
"sqlbuild-analysis",
2123
"sqlbuild-rules",
2224
"sqlbuild-python",
@@ -64,6 +66,7 @@ sha2 = "0.10"
6466
sqlbuild-analysis = { path = "crates/sqlbuild-analysis" }
6567
sqlbuild-config = { path = "crates/sqlbuild-config" }
6668
sqlbuild-core = { path = "crates/sqlbuild-core" }
69+
sqlbuild-discovery = { path = "crates/sqlbuild-discovery" }
6770
sqlbuild-rules = { path = "crates/sqlbuild-rules" }
6871
sqlbuild-sqltext = { path = "crates/sqlbuild-sqltext" }
6972
stacker = "0.1"

‎crates/README.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ that can be tested and benchmarked with `cargo test` alone.
99
| `sqlbuild-core` | Constants shared by several crates, the panic boundary that turns an unwinding native failure into a named compiler error, passive identity, diagnostic and SQL value types, text positions, and a JSON emitter whose output matches Python's `json.dumps` and `orjson` byte for byte. |
1010
| `sqlbuild-sqltext` | Lexical SQL text without polyglot: comment, quote and parenthesis scanning, the rules quote policy, model header tokenization and matching, static variable substitution and static reference extraction. |
1111
| `sqlbuild-config` | Configuration file reading without Python: `tomllib`-compatible TOML, PyYAML `safe_load`-compatible YAML 1.1, and typed project and local config readers for the fields discovery needs. Any error means "defer to Python", which re-parses for exact messages. |
12+
| `sqlbuild-discovery` | Native project discovery: the shared directory walk with Python's glob and sort semantics, file reading with Python's newline handling, and parsing of authored files with Python's exact discovery messages. Results are plain data the Python discovery facade materialises; anything it cannot reproduce exactly defers to Python. |
1213
| `sqlbuild-analysis` | SQL analysis over polyglot: SQL tokens, query analysis, the binding catalog, semantic validation and usage, column references, and SQL-test extraction, planning and rendering. |
1314
| `sqlbuild-rules` | Built-in rules and the rules engine, the custom-rule host, SQL lint, quality checks and formatting, rules configuration and the request models. It also owns the build identity script. |
1415
| `sqlbuild-python` | The only PyO3 crate: the `_native` module, its Python classes and functions, conversions from Python objects, and the process allocator. |
@@ -19,7 +20,8 @@ Dependencies point one way, from the top of this graph to the bottom:
1920

2021
```text
2122
sqlbuild-python
22-
-> sqlbuild-rules -> sqlbuild-analysis -> sqlbuild-config -> sqlbuild-sqltext -> sqlbuild-core
23+
-> sqlbuild-rules -> sqlbuild-analysis -> sqlbuild-discovery -> sqlbuild-config
24+
-> sqlbuild-sqltext -> sqlbuild-core
2325
```
2426

2527
`sqlbuild-config` does not depend on the crates below it today; its place in the order only

0 commit comments

Comments
 (0)