Repository navigation
fix(compiler)!: reject unknown config keys and keep scenario SQL as authored - #906
Merged
Merged
Conversation
MODEL() headers, [defaults], [path_defaults.*] and sources/*.yml silently ignored unknown keys, so a typo or a removed setting had no effect. Every statement header (MODEL, FUNCTION, AUDIT, TEST, HOOK, SCENARIO, @udf) now shares one unsupported-keys error that names the key with its file and line and suggests the nearest supported key. Custom materialization values stay inside the free-form config block. BREAKING CHANGE: unknown keys in MODEL() headers, project defaults, path defaults and sources files now fail compilation.
…odels One model with thousands of diagnostics compiled in quadratic time: locating each unknown column rescanned the authored SQL natively, and diagnostic recovery and explanation rebuilt per-model column sets, lineage lists, line splits and binary-expression scans for every diagnostic. Index each once per model and look positions up by bisect or hash. A one-model scaling guard compares 1k and 4k diagnostics.
When the layout oracle rewrote a substituted region, for example DuckDB
'north' 'east' printed as CONCAT('north', 'east'), the token after the
region took the oracle's spacing after the CONCAT( it introduced, so
`replace, 'north'` became `replace,'north'` and lost its line break. The
spacing now comes from the oracle's own copy of the authored token that
precedes it.
sqb dbt run and build enforced only the missing-origin and retention decrease policies before executing SQLBuild work, so a table_type_downgrade, snapshot full-refresh or max_models policy could be bypassed through the dbt interop path. Both commands now share the sqb build gate sequence. sqb dbt has no allow flags: a policy that needs confirmation prompts on a terminal and otherwise refuses with guidance naming the matching sqb build flag or policy setting.
Scenario expected, assertion, helper and fixture SQL was parsed and regenerated with the generic dialect to swap relation markers, so dialect syntax was rewritten or rejected (S506) and the executed SQL differed from what was authored. Markers are now replaced in the authored text, skipping comments and quoted text under the project adapter's lexical rules, also for --local replay: backslash escapes on Snowflake, BigQuery and Databricks, E'...' strings on DuckDB and Postgres, nested block comments where the dialect nests them, and BigQuery '#' and Snowflake '//' line comments. A marker the scenario cannot resolve, or SQL with an unclosed string or comment, fails planning with S511 naming the scenario file instead of being left in the executed SQL. Local replay no longer regenerates SQL captured from DuckDB.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Several compile paths silently accepted mistakes or misbehaved: unknown header and config keys were ignored (CHI-458), one model with thousands of diagnostics compiled quadratically (CHI-457), the formatter dropped a space after a comma before adjacent string literals (CHI-456),
sqb dbt run/buildskippedsqb build's plan safety policies (CHI-451), and scenario SQL was regenerated, rewriting or rejecting dialect syntax (CHI-454).Changes
MODEL()and other resource headers fail with D002 (nearest-key help); unknown[defaults]/[path_defaults]keys fail with D001; unknownsources/*.ymlkeys fail with D006. Docs list the supported[defaults]keys.sqb dbt run/buildapply the full build policy sequence (table type, snapshot full refresh,max_models): prompt on a TTY, otherwise refuse.--local); an unresolvable marker fails with S511 naming the file.Verification
make test,make check-ci; one independent review plus a follow-up.