Repository navigation
refactor(native): delete the Python discovery implementation - #1007
Merged
Merged
Conversation
…discovery Scope inspection, selected contract discovery, model description inputs and SQL test fixture formatting now read model, SQL test, scenario and source files and the declaration layout through the native discovery instead of the Python discovery helpers. Native discovery gains model name and test path selection, per-file fault reporting, in-memory SQL test and scenario parsing, and declaration scans isolated to one kind, so these callers keep their exact outcomes. The Python engine's discovery stage is unchanged and remains the differential oracle.
Model, SQL test, scenario, source and schema files and the declaration layout are now discovered natively under both compiler engines, with no Python fallback: - Unreadable files raise Python's own read errors natively: the exact UnicodeDecodeError for invalid UTF-8 and OSError for refused reads, with Windows error codes mapped the way Python maps them. - Invalid YAML in sources files (D006) and seed declarations or schema.yml (D005) names the file, line and column. YAML outside the supported plain subset (tabs, directives, Python-only tags, recursive aliases and similar) fails with the same codes and a hint. - A model, SQL test, scenario, source or schema file whose path is not UTF-8 fails with D016, showing invalid bytes as \xNN escapes; callers that tolerate bad files report it per file. Other files and directories with such names are ignored and declaration files are read by their real names, as before; every name keeps its own identity even when names share a lossy spelling. A non-UTF-8 project directory fails. - Native discovery runs on Windows: paths print with backslashes, sort case-insensitively as Python's Windows paths do, and junctions are walked like directories. A Windows CI job covers it. - An unknown Python or Unicode version fails with D017 naming the supported Python versions. The discovery fact cache, which only served the Python fallbacks, is removed.
Model, SQL test, scenario, source and schema file discovery and the declaration layout scan now exist only in native discovery, for both compiler engines. Delete the Python walkers, parsers, YAML loaders and the discovery fact cache they used, keep the Python semantic layers that build discovered objects from native results, and point the remaining behaviour tests at native discovery. Parity oracles that compared the two implementations are removed with the Python side.
Native discovery ships Unicode and text tables for Python 3.12, 3.13 and 3.14 and refuses other versions, so declare that range in requires-python and the lockfile, and say "Python 3.12–3.14" in the quickstart and contributing guide.
kvlonge
enabled auto-merge (squash)
October 7, 2026 16:19
…aw names A raw-segment test built its names from POSIX byte hex, which Windows decodes as UTF-16 units, so it failed there. The product round-trips Windows names correctly; only the display was off: a lone surrogate in U+DC80-U+DCFF was shown as a \xNN byte. Windows now always shows lone surrogates as \uXXXX, in Rust and in Python's own messages. Decoding and display take the platform as a parameter so Linux tests cover both the POSIX and the Windows branches, and a Windows-only test walks real files whose names hold lone surrogates.
When stdout or stderr was opened with a non-UTF-8 encoding, such as the Windows cp1252 code page used for pipes and redirects, sqb crashed with E001 on the first non-ASCII character of its summary output. The CLI entry point now reconfigures any non-UTF-8 standard stream to UTF-8 with backslashreplace, so output never fails to encode and JSON output stays valid. Library use and the rule-engine host process are unaffected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Native discovery is the default for model, test, scenario, source and schema files and the declaration layout. A second Python copy doubles upkeep and drifts.
Changes
D017.D016, invalid bytes shown as\xNN(per file where tolerated); other such names are ignoredD006(sources) orD005(schema/seed files) with line/column.%YAML/%TAGdirectives,!!binary/!!set/!!omap/!!pairs, an empty!tag, recursive aliases, aliases expanding past the size limit, nesting deeper than 256, integers over 4300 digits, implicit keys over 1000 characters, NEL/LS/PS line breaks,?in a flow plain scalar,:right before,/]/}in flow context, block scalars as keys, in flow collections or with ambiguous indentation.junctioncrate) and Python's read-error codes. Matching stays case-sensitive. New Windows CI job.SQLBUILD_COMPILER_ENGINE=pythondiscovers these natively.requires-python = ">=3.12,<3.15", lockfile and docs.Verification
cargo clippy/cargo test,make check-native-layering,make check-ci,make test, actionlintmake compiler-differential: 147 projects identical; its e2e tests pass