Repository navigation
ci: benchmark dbt v2 partial parse and dirty compiles - #1189
Merged
Merged
Conversation
kvlonge
enabled auto-merge (squash)
October 21, 2026 03:06
kvlonge
disabled auto-merge
October 21, 2026 03:37
kvlonge
enabled auto-merge (squash)
October 21, 2026 03:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
dbt v2 can cache between runs when asked (
--partial-parse, hidden in 2.0.9;--partial-load;--dirtycompiles only changed models and descendants), so timing only its default is unfair. Strict analysis is only in thedbtdistribution and can fall back silently, so the workflow must prove it ran. dbt Core is a reference only.Changes
bench.sh: dbt v2 cached,--dirtyand default variants, each also strict.v2_compile_strict_remotereads source schemas from the warehouse;sqb_norules_fulladded..cmdper run.write_dbt_dense.py: sources declareschema_origin: local(like SQLBuild);dbt-remote/keeps dbt's default.warehouse.duckdbholds every source and model relation, empty, typed, matching materialization.check_strict.py, run by the workflow after correctness (before the benchmark) and after it: the seeded unknown column fails strict with dbt0227 in default, remote, cached and--dirtymodes; plain compile passes it; clean passes strict; every timed strict run exits 0; no strict output matches the listed fallback patterns.summarize.py: labelled rows (SQLBuild default first; dbt v2 cached,--dirty, default, strict alongside), "warm (replay)", full-parse fallback and skipped analysis noted.core_*.Verification
check_strict.py correctnesspass; a fake fallback line or a passing strict run fails it.check_strict.py runspasses; local sources download no schemas.sqb compileoutput identical without the warehouse.actionlint,ruff, hygiene scan pass.