diff --git a/.github/demos/rename.gif b/.github/demos/rename.gif index 98be0cb6cf..779271d99d 100644 Binary files a/.github/demos/rename.gif and b/.github/demos/rename.gif differ diff --git a/.github/demos/rules.gif b/.github/demos/rules.gif index f3bca77c61..d26d0e56b4 100644 Binary files a/.github/demos/rules.gif and b/.github/demos/rules.gif differ diff --git a/website/public/assets/og-sqlbuild.png b/website/public/assets/og-sqlbuild.png index 36b0b101e9..a66d1548cd 100644 Binary files a/website/public/assets/og-sqlbuild.png and b/website/public/assets/og-sqlbuild.png differ diff --git a/website/public/index.html b/website/public/index.html index d17b4d2835..9216687f79 100644 --- a/website/public/index.html +++ b/website/public/index.html @@ -18,7 +18,7 @@ - + @@ -77,7 +77,7 @@
sqb rename and sqb mv update every reference and keep the existing table, so an incremental model doesn't start again. The old name keeps working as a view. Renamed columns keep their history too.
-sqb rename model:daily_activity_rollup daily_order_rollup +sqb rename daily_activity_rollup daily_order_rollup Rename model daily_activity_rollup -> daily_order_rollup ├── models/marts/daily_order_rollup.sql (moved from models/marts/daily_activity_rollup.sql) +│ └── 1:8 migration + migrate_from daily_activity_rollup └── models/marts/hourly_activity_with_daily_context.sql - └── 36:19 reference __ref("daily_activity_rollup") -> __ref("daily_order_rollup") + ├── 11:5 header daily_activity_rollup -> daily_order_rollup + └── 45:21 reference __ref("daily_activity_rollup") -> __ref("daily_order_rollup") Compiled: ok, 2 files changed sqb plan @@ -432,7 +434,7 @@diff --git a/website/public/styles.css b/website/public/styles.css index 64813f0145..36c40f6163 100644 --- a/website/public/styles.css +++ b/website/public/styles.css @@ -584,6 +584,7 @@ h3 { .start .terminal { border: 1px solid var(--panel-border); border-radius: var(--radius); + white-space: pre-wrap; } .adapters { diff --git a/website/recordings/prepare.sh b/website/recordings/prepare.sh index b3febac79a..444d238c2e 100755 --- a/website/recordings/prepare.sh +++ b/website/recordings/prepare.sh @@ -39,10 +39,12 @@ open(path, "w").write(text) PY ;; rules) - # Make a mart read a raw source directly, which the project's layering rule forbids. + # Make a mart read a raw source directly, which the project's layering rule forbids. The raw + # source calls the column status, so alias it to keep the mart's columns valid. cp -r "$examples/waffle-shop" "$dir" cd "$dir" - sed -i 's/LEFT JOIN __ref("stg_payments") p/LEFT JOIN __source("raw__payments") p/' \ + sed -i -e 's/LEFT JOIN __ref("stg_payments") p/LEFT JOIN __source("raw__payments") p/' \ + -e 's/ p\.payment_status,/ p.status AS payment_status,/' \ models/marts/fact_orders.sql ;; scope) cp -r "$examples/waffle-shop" "$dir" diff --git a/website/recordings/tapes/rename.tape b/website/recordings/tapes/rename.tape index f0a3cc0cc0..b40353dd34 100644 --- a/website/recordings/tapes/rename.tape +++ b/website/recordings/tapes/rename.tape @@ -10,7 +10,7 @@ Show Type "tree --noreport models" Enter Sleep 1500ms -Type "sqb mv model:revenue models/finance/daily_revenue.sql" +Type "sqb mv revenue models/finance/daily_revenue.sql" Sleep 300ms Enter Wait@60s /\$\s*$/ diff --git a/website/src/content/docs/docs/cli/build.mdx b/website/src/content/docs/docs/cli/build.mdx index 57db524614..7dab358b73 100644 --- a/website/src/content/docs/docs/cli/build.mdx +++ b/website/src/content/docs/docs/cli/build.mdx @@ -71,22 +71,27 @@ audit reads, so the audit can gate the resource like its other audits. See ``` Execution sqb build (concurrency: 1) - 1/13 seed waffle_types OK 0.09s - 2/13 view stg_customers OK 0.05s - audit not_null (customer_id) PASS - audit unique (customer_id) PASS - 3/13 view stg_orders OK 0.03s + 1/17 source raw__customers OK 0.05s rows=5 + 2/17 source raw__orders OK 0.03s rows=10 + + 3/17 seed waffle_types OK 0.03s + 7/17 view stg_orders OK 0.04s test test_stg_orders PASS + expect expected stg_orders PASS + expect assertion order_ids_are_not_null PASS audit not_null (order_id) PASS audit unique (order_id) PASS - 10/13 table hourly_order_activity (delete_insert) OK 0.16s - audit (d) expression_is_true PASS 4/4 - audit (d) not_null (activity_hour) PASS 4/4 - audit (f) expression_is_true PASS + audit not_null (customer_id) PASS + audit relationships (waffle_type_id) PASS + audit accepted_values (status) PASS + 13/17 table hourly_order_activity (delete_insert) OK 0.24s + 5 batches (1d) range 2026-03-31T09:00:00 → 2026-04-04T14:00:00 8 rows + audit (d) orders_placed_is_non_negative PASS 5/5 + audit (d) not_null (activity_hour) PASS 5/5 + audit (f) orders_placed_is_non_negative PASS audit (f) not_null (activity_hour) PASS -Completed successfully. -PASS=66 WARN=0 FAIL=0 SKIP=0 TOTAL=66 (1.09s) +✓ Completed successfully PASS=99 WARN=0 FAIL=0 INSUFFICIENT=0 SKIP=0 TOTAL=99 (2.13s) ``` ## Deferred builds diff --git a/website/src/content/docs/docs/quickstart.mdx b/website/src/content/docs/docs/quickstart.mdx index 303aae4010..3d6cfffde1 100644 --- a/website/src/content/docs/docs/quickstart.mdx +++ b/website/src/content/docs/docs/quickstart.mdx @@ -36,29 +36,37 @@ sqb plan ``` ``` -Plan ready (13 selected) +Plan ready 15 selected, 2 sources to load + +Sources to load (2) +├── raw__customers table +└── raw__orders delete_insert (cursor: ordered_at) + +Changed functions (2) +├── table_fn__customer_orders table function +└── udf__is_completed_order sql udf First run (12) - stg_customers view - stg_payments view - stg_orders view - daily_order_partitioned partition_tracked (custom) - daily_revenue table - dim_customers table - fact_orders table - customer_status_snapshot merge (timestamp) - hourly_order_activity delete_insert (timestamp, microbatch) - daily_activity_rollup delete_insert (timestamp, microbatch) - hourly_activity_with_daily_context delete_insert (timestamp, microbatch) - order_status_index delete_insert (integer) +├── stg_customers view +├── stg_payments view +├── stg_orders view +├── daily_order_partitioned partition_tracked (custom) +├── daily_revenue table +├── dim_customers table +├── fact_orders table +├── customer_status_snapshot merge (timestamp) +├── hourly_order_activity delete_insert (timestamp, microbatch) +├── daily_activity_rollup delete_insert (timestamp, microbatch) +├── hourly_activity_with_daily_context delete_insert (timestamp, microbatch) +└── order_status_index delete_insert (integer) Seeds (1) - waffle_types +└── waffle_types (first_run) -✓ Plan complete 13 selected +✓ Plan complete 15 selected, 2 sources to load ``` -Source data is defined as inline expressions in `sources/raw.yml`, so SQLBuild resolves everything at compile time - no seeding step needed. +`raw__customers` and `raw__orders` are loaded by a Python loader (`sources/loaders/waffle_sources.py`) at the start of the build; `raw__payments` is an inline expression in `sources/raw.yml`. Incremental models also list their cursor and window under their row; they are left out here. ## 3. Build @@ -73,55 +81,60 @@ SQLBuild materializes all models in DAG order. SQL unit tests run before their t ``` Execution sqb build (concurrency: 1) - 1/13 seed waffle_types OK 0.09s - 2/13 view stg_customers OK 0.05s + 1/17 source raw__customers OK 0.05s rows=5 + 2/17 source raw__orders OK 0.03s rows=10 + + 3/17 seed waffle_types OK 0.03s + 4/17 view stg_customers OK 0.02s audit not_null (customer_id) PASS audit unique (customer_id) PASS audit not_null (email) PASS - 3/13 view stg_payments OK 0.02s - audit not_null (payment_id) PASS - audit unique (payment_id) PASS - audit not_null (order_id) PASS - 4/13 view stg_orders OK 0.03s + 7/17 view stg_orders OK 0.04s + test test_daily_revenue_chain PASS + expect expected stg_orders PASS + expect expected stg_payments PASS + expect expected daily_revenue PASS + expect assertion daily_revenue_is_non_negative PASS test test_stg_orders PASS + expect expected stg_orders PASS + expect assertion order_ids_are_not_null PASS audit not_null (order_id) PASS audit unique (order_id) PASS audit not_null (customer_id) PASS + audit relationships (waffle_type_id) PASS audit accepted_values (status) PASS - 5/13 custom daily_order_partitioned OK 0.17s - audit expression_is_true PASS 4/4 + 8/17 custom daily_order_partitioned OK 0.21s + audit waffles_ordered_is_positive PASS 4/4 audit not_null (order_date) PASS 4/4 - 6/13 table daily_revenue OK 0.05s - audit expression_is_true PASS - 7/13 table dim_customers OK 0.05s - audit not_null (customer_id) PASS - audit unique (customer_id) PASS - 8/13 table fact_orders OK 0.04s + 9/17 table daily_revenue OK 0.07s + test test_daily_revenue_chain PASS + expect expected stg_orders PASS + expect expected stg_payments PASS + expect expected daily_revenue PASS + expect assertion daily_revenue_is_non_negative PASS + audit revenue_is_non_negative PASS + 11/17 table fact_orders OK 0.05s + test test_fact_orders PASS + expect expected fact_orders PASS + expect assertion line_totals_are_non_negative PASS audit not_null (order_id) PASS - 9/13 table customer_status_snapshot (merge) OK 0.04s + 12/17 table customer_status_snapshot (merge) OK 0.05s audit not_null (customer_id) PASS audit unique (customer_id) PASS - 10/13 table hourly_order_activity (delete_insert) OK 0.16s - audit (d) expression_is_true PASS 4/4 - audit (d) not_null (activity_hour) PASS 4/4 - audit (f) expression_is_true PASS + 13/17 table hourly_order_activity (delete_insert) OK 0.24s + 5 batches (1d) range 2026-03-31T09:00:00 → 2026-04-04T14:00:00 8 rows + audit (d) orders_placed_is_non_negative PASS 5/5 + audit (d) not_null (activity_hour) PASS 5/5 + audit (f) orders_placed_is_non_negative PASS audit (f) not_null (activity_hour) PASS - 11/13 table order_status_index (delete_insert) OK 0.03s - audit not_null (order_id) PASS - audit unique (order_id) PASS - 12/13 table daily_activity_rollup (delete_insert) OK 0.08s - audit (d) expression_is_true PASS 2/2 - audit (d) not_null (activity_day) PASS 2/2 - audit (f) expression_is_true PASS - audit (f) not_null (activity_day) PASS - 13/13 table hourly_activity_with_daily_context (delete_insert) OK 0.17s - audit (d) expression_is_true PASS 4/4 - audit (d) not_null (activity_hour) PASS 4/4 - audit (f) expression_is_true PASS + 17/17 table hourly_activity_with_daily_context (delete_insert) OK 0.63s + 13 batches (6h) range 2026-03-31T18:00:00 → 2026-04-03T23:00:00 0 rows + audit (d) day_orders_cover_hourly_orders PASS 13/13 + audit (d) not_null (activity_hour) PASS 13/13 + audit (f) day_orders_cover_hourly_orders PASS audit (f) not_null (activity_hour) PASS -Completed successfully. -PASS=66 WARN=0 FAIL=0 SKIP=0 TOTAL=66 (1.09s) +✓ Completed successfully PASS=99 WARN=0 FAIL=0 INSUFFICIENT=0 SKIP=0 TOTAL=99 (2.13s) ``` Notice `audit (d)` and `audit (f)` on incremental models - these are delta-phase and final-phase audits. Delta audits validate each batch before DML is applied; final audits run against the target after promotion. @@ -135,23 +148,37 @@ sqb plan ``` ``` -Plan ready (13 selected) - -Normal (12) - 3 view - 3 table - 3 delete_insert (timestamp, microbatch) - 1 partition_tracked (custom) - 1 merge (timestamp) - 1 delete_insert (integer) +Plan ready 15 selected, 2 sources to load + +Sources to load (2) +├── raw__customers table +└── raw__orders delete_insert (cursor: ordered_at) + +Models (12) +├── stg_customers view +├── stg_payments view +├── stg_orders view +├── daily_order_partitioned partition_tracked (custom) +├── daily_revenue table +├── dim_customers table +├── fact_orders table +├── customer_status_snapshot merge (timestamp) +├── hourly_order_activity delete_insert (timestamp, microbatch) +├── daily_activity_rollup delete_insert (timestamp, microbatch) +├── hourly_activity_with_daily_context delete_insert (timestamp, microbatch) +└── order_status_index delete_insert (integer) + +Functions (2) +├── table_fn__customer_orders table function +└── udf__is_completed_order sql udf Seeds (1) - waffle_types +└── waffle_types (current) -✓ Plan complete 13 selected +✓ Plan complete 15 selected, 2 sources to load ``` -All models now show `Normal` instead of `First run`. +The models and functions are no longer listed as first run, and the seed is current. The two loader-managed sources load on every build. ## 5. Common operationsStart simple
sqb playground waffle-shop cd waffle-shop sqb build -✓ Completed successfully PASS=95 FAIL=0 +✓ Completed successfully PASS=99 FAIL=0 TOTAL=99 (2.13s) sqb test