Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified .github/demos/rename.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified .github/demos/rules.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/public/assets/og-sqlbuild.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
12 changes: 7 additions & 5 deletions website/public/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<meta property="og:image" content="https://sqlbuild.com/assets/og-sqlbuild.png" />
<meta property="og:image:width" content="1200" />
<meta property="og:image:height" content="630" />
<meta property="og:image:alt" content="SQLBuild: the refactorable warehouse. sqb rename renames a model, and sqb plan migrates its table from prod.revenue to prod.daily_revenue. History kept. Nothing rebuilt." />
<meta property="og:image:alt" content="SQLBuild: the refactorable warehouse. sqb rename revenue daily_revenue renames a model, and sqb plan migrates its table from prod.revenue to prod.daily_revenue. History kept. Nothing rebuilt." />
<meta name="twitter:card" content="summary_large_image" />
<meta name="twitter:image" content="https://sqlbuild.com/assets/og-sqlbuild.png" />
<link rel="icon" type="image/png" href="assets/icon.png" />
Expand Down Expand Up @@ -77,7 +77,7 @@ <h1>The refactorable warehouse</h1>
class="terminal hero-visual"
role="img"
aria-label="sqb mv moves and renames a model file, and sqb plan shows a migration of the existing table instead of a rebuild, with a view keeping the old name working."
><span class="hv-l hv-l0"><span class="cmd">sqb mv model:revenue<span class="hv-break"> </span>models/finance/daily_revenue.sql</span></span><span class="hv-stack"><span class="hv-a"><span class="tree"><span class="dim">models</span>
><span class="hv-l hv-l0"><span class="cmd">sqb mv revenue<span class="hv-break"> </span>models/finance/daily_revenue.sql</span></span><span class="hv-stack"><span class="hv-a"><span class="tree"><span class="dim">models</span>
<span class="dim">└──</span> marts
<span class="dim">└──</span> revenue.sql</span></span><span class="hv-b"><span class="tree"><span class="dim">models</span>
<span class="dim">└──</span> finance
Expand Down Expand Up @@ -327,11 +327,13 @@ <h3>Rename models without rebuilding history</h3>
<code>sqb rename</code> and <code>sqb mv</code> update every reference and keep the existing table, so an incremental model doesn't start again. The old name keeps working as a view. Renamed columns keep their history too.
</p>
</div>
<pre class="terminal"><span class="cmd">sqb rename model:daily_activity_rollup daily_order_rollup</span>
<pre class="terminal"><span class="cmd">sqb rename daily_activity_rollup daily_order_rollup</span>
<span class="info strong">Rename model daily_activity_rollup -&gt; daily_order_rollup</span>
<span class="dim">├──</span> models/marts/daily_order_rollup.sql<span class="dim"> (moved from models/marts/daily_activity_rollup.sql)</span>
<span class="dim">│ └──</span> 1:8 migration + migrate_from daily_activity_rollup
<span class="dim">└──</span> models/marts/hourly_activity_with_daily_context.sql
<span class="dim"> └──</span> 36:19 reference __ref(&quot;daily_activity_rollup&quot;) -&gt; __ref(&quot;daily_order_rollup&quot;)
<span class="dim"> ├──</span> 11:5 header daily_activity_rollup -&gt; daily_order_rollup
<span class="dim"> └──</span> 45:21 reference __ref(&quot;daily_activity_rollup&quot;) -&gt; __ref(&quot;daily_order_rollup&quot;)
<span class="ok strong">Compiled: ok, 2 files changed</span>

<span class="cmd">sqb plan</span>
Expand Down Expand Up @@ -432,7 +434,7 @@ <h2>Start simple</h2>
<span class="cmd">sqb playground waffle-shop</span>
<span class="cmd">cd waffle-shop</span>
<span class="cmd">sqb build</span>
<span class="ok">✓ Completed successfully</span> PASS=95 FAIL=0
<span class="ok">✓ Completed successfully</span> PASS=99 FAIL=0 TOTAL=99 <span class="dim">(2.13s)</span>
<span class="cmd">sqb test</span></pre>
</div>
</section>
Expand Down
1 change: 1 addition & 0 deletions website/public/styles.css
Original file line number Diff line number Diff line change
Expand Up @@ -584,6 +584,7 @@ h3 {
.start .terminal {
border: 1px solid var(--panel-border);
border-radius: var(--radius);
white-space: pre-wrap;
}

.adapters {
Expand Down
6 changes: 4 additions & 2 deletions website/recordings/prepare.sh
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,12 @@ open(path, "w").write(text)
PY
;;
rules)
# Make a mart read a raw source directly, which the project's layering rule forbids.
# Make a mart read a raw source directly, which the project's layering rule forbids. The raw
# source calls the column status, so alias it to keep the mart's columns valid.
cp -r "$examples/waffle-shop" "$dir"
cd "$dir"
sed -i 's/LEFT JOIN __ref("stg_payments") p/LEFT JOIN __source("raw__payments") p/' \
sed -i -e 's/LEFT JOIN __ref("stg_payments") p/LEFT JOIN __source("raw__payments") p/' \
-e 's/ p\.payment_status,/ p.status AS payment_status,/' \
models/marts/fact_orders.sql ;;
scope)
cp -r "$examples/waffle-shop" "$dir"
Expand Down
2 changes: 1 addition & 1 deletion website/recordings/tapes/rename.tape
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Show
Type "tree --noreport models"
Enter
Sleep 1500ms
Type "sqb mv model:revenue models/finance/daily_revenue.sql"
Type "sqb mv revenue models/finance/daily_revenue.sql"
Sleep 300ms
Enter
Wait@60s /\$\s*$/
Expand Down
27 changes: 16 additions & 11 deletions website/src/content/docs/docs/cli/build.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -71,22 +71,27 @@ audit reads, so the audit can gate the resource like its other audits. See
```
Execution sqb build (concurrency: 1)

1/13 seed waffle_types OK 0.09s
2/13 view stg_customers OK 0.05s
audit not_null (customer_id) PASS
audit unique (customer_id) PASS
3/13 view stg_orders OK 0.03s
1/17 source raw__customers OK 0.05s rows=5
2/17 source raw__orders OK 0.03s rows=10

3/17 seed waffle_types OK 0.03s
7/17 view stg_orders OK 0.04s
test test_stg_orders PASS
expect expected stg_orders PASS
expect assertion order_ids_are_not_null PASS
audit not_null (order_id) PASS
audit unique (order_id) PASS
10/13 table hourly_order_activity (delete_insert) OK 0.16s
audit (d) expression_is_true PASS 4/4
audit (d) not_null (activity_hour) PASS 4/4
audit (f) expression_is_true PASS
audit not_null (customer_id) PASS
audit relationships (waffle_type_id) PASS
audit accepted_values (status) PASS
13/17 table hourly_order_activity (delete_insert) OK 0.24s
5 batches (1d) range 2026-03-31T09:00:00 → 2026-04-04T14:00:00 8 rows
audit (d) orders_placed_is_non_negative PASS 5/5
audit (d) not_null (activity_hour) PASS 5/5
audit (f) orders_placed_is_non_negative PASS
audit (f) not_null (activity_hour) PASS

Completed successfully.
PASS=66 WARN=0 FAIL=0 SKIP=0 TOTAL=66 (1.09s)
✓ Completed successfully PASS=99 WARN=0 FAIL=0 INSUFFICIENT=0 SKIP=0 TOTAL=99 (2.13s)
```

## Deferred builds
Expand Down
151 changes: 89 additions & 62 deletions website/src/content/docs/docs/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -36,29 +36,37 @@ sqb plan
```

```
Plan ready (13 selected)
Plan ready 15 selected, 2 sources to load

Sources to load (2)
├── raw__customers table
└── raw__orders delete_insert (cursor: ordered_at)

Changed functions (2)
├── table_fn__customer_orders table function
└── udf__is_completed_order sql udf

First run (12)
stg_customers view
stg_payments view
stg_orders view
daily_order_partitioned partition_tracked (custom)
daily_revenue table
dim_customers table
fact_orders table
customer_status_snapshot merge (timestamp)
hourly_order_activity delete_insert (timestamp, microbatch)
daily_activity_rollup delete_insert (timestamp, microbatch)
hourly_activity_with_daily_context delete_insert (timestamp, microbatch)
order_status_index delete_insert (integer)
├── stg_customers view
├── stg_payments view
├── stg_orders view
├── daily_order_partitioned partition_tracked (custom)
├── daily_revenue table
├── dim_customers table
├── fact_orders table
├── customer_status_snapshot merge (timestamp)
├── hourly_order_activity delete_insert (timestamp, microbatch)
├── daily_activity_rollup delete_insert (timestamp, microbatch)
├── hourly_activity_with_daily_context delete_insert (timestamp, microbatch)
└── order_status_index delete_insert (integer)

Seeds (1)
waffle_types
└── waffle_types (first_run)

✓ Plan complete 13 selected
✓ Plan complete 15 selected, 2 sources to load
```

Source data is defined as inline expressions in `sources/raw.yml`, so SQLBuild resolves everything at compile time - no seeding step needed.
`raw__customers` and `raw__orders` are loaded by a Python loader (`sources/loaders/waffle_sources.py`) at the start of the build; `raw__payments` is an inline expression in `sources/raw.yml`. Incremental models also list their cursor and window under their row; they are left out here.

## 3. Build

Expand All @@ -73,55 +81,60 @@ SQLBuild materializes all models in DAG order. SQL unit tests run before their t
```
Execution sqb build (concurrency: 1)

1/13 seed waffle_types OK 0.09s
2/13 view stg_customers OK 0.05s
1/17 source raw__customers OK 0.05s rows=5
2/17 source raw__orders OK 0.03s rows=10

3/17 seed waffle_types OK 0.03s
4/17 view stg_customers OK 0.02s
audit not_null (customer_id) PASS
audit unique (customer_id) PASS
audit not_null (email) PASS
3/13 view stg_payments OK 0.02s
audit not_null (payment_id) PASS
audit unique (payment_id) PASS
audit not_null (order_id) PASS
4/13 view stg_orders OK 0.03s
7/17 view stg_orders OK 0.04s
test test_daily_revenue_chain PASS
expect expected stg_orders PASS
expect expected stg_payments PASS
expect expected daily_revenue PASS
expect assertion daily_revenue_is_non_negative PASS
test test_stg_orders PASS
expect expected stg_orders PASS
expect assertion order_ids_are_not_null PASS
audit not_null (order_id) PASS
audit unique (order_id) PASS
audit not_null (customer_id) PASS
audit relationships (waffle_type_id) PASS
audit accepted_values (status) PASS
5/13 custom daily_order_partitioned OK 0.17s
audit expression_is_true PASS 4/4
8/17 custom daily_order_partitioned OK 0.21s
audit waffles_ordered_is_positive PASS 4/4
audit not_null (order_date) PASS 4/4
6/13 table daily_revenue OK 0.05s
audit expression_is_true PASS
7/13 table dim_customers OK 0.05s
audit not_null (customer_id) PASS
audit unique (customer_id) PASS
8/13 table fact_orders OK 0.04s
9/17 table daily_revenue OK 0.07s
test test_daily_revenue_chain PASS
expect expected stg_orders PASS
expect expected stg_payments PASS
expect expected daily_revenue PASS
expect assertion daily_revenue_is_non_negative PASS
audit revenue_is_non_negative PASS
11/17 table fact_orders OK 0.05s
test test_fact_orders PASS
expect expected fact_orders PASS
expect assertion line_totals_are_non_negative PASS
audit not_null (order_id) PASS
9/13 table customer_status_snapshot (merge) OK 0.04s
12/17 table customer_status_snapshot (merge) OK 0.05s
audit not_null (customer_id) PASS
audit unique (customer_id) PASS
10/13 table hourly_order_activity (delete_insert) OK 0.16s
audit (d) expression_is_true PASS 4/4
audit (d) not_null (activity_hour) PASS 4/4
audit (f) expression_is_true PASS
13/17 table hourly_order_activity (delete_insert) OK 0.24s
5 batches (1d) range 2026-03-31T09:00:00 → 2026-04-04T14:00:00 8 rows
audit (d) orders_placed_is_non_negative PASS 5/5
audit (d) not_null (activity_hour) PASS 5/5
audit (f) orders_placed_is_non_negative PASS
audit (f) not_null (activity_hour) PASS
11/13 table order_status_index (delete_insert) OK 0.03s
audit not_null (order_id) PASS
audit unique (order_id) PASS
12/13 table daily_activity_rollup (delete_insert) OK 0.08s
audit (d) expression_is_true PASS 2/2
audit (d) not_null (activity_day) PASS 2/2
audit (f) expression_is_true PASS
audit (f) not_null (activity_day) PASS
13/13 table hourly_activity_with_daily_context (delete_insert) OK 0.17s
audit (d) expression_is_true PASS 4/4
audit (d) not_null (activity_hour) PASS 4/4
audit (f) expression_is_true PASS
17/17 table hourly_activity_with_daily_context (delete_insert) OK 0.63s
13 batches (6h) range 2026-03-31T18:00:00 → 2026-04-03T23:00:00 0 rows
audit (d) day_orders_cover_hourly_orders PASS 13/13
audit (d) not_null (activity_hour) PASS 13/13
audit (f) day_orders_cover_hourly_orders PASS
audit (f) not_null (activity_hour) PASS

Completed successfully.
PASS=66 WARN=0 FAIL=0 SKIP=0 TOTAL=66 (1.09s)
✓ Completed successfully PASS=99 WARN=0 FAIL=0 INSUFFICIENT=0 SKIP=0 TOTAL=99 (2.13s)
```

Notice `audit (d)` and `audit (f)` on incremental models - these are delta-phase and final-phase audits. Delta audits validate each batch before DML is applied; final audits run against the target after promotion.
Expand All @@ -135,23 +148,37 @@ sqb plan
```

```
Plan ready (13 selected)

Normal (12)
3 view
3 table
3 delete_insert (timestamp, microbatch)
1 partition_tracked (custom)
1 merge (timestamp)
1 delete_insert (integer)
Plan ready 15 selected, 2 sources to load

Sources to load (2)
├── raw__customers table
└── raw__orders delete_insert (cursor: ordered_at)

Models (12)
├── stg_customers view
├── stg_payments view
├── stg_orders view
├── daily_order_partitioned partition_tracked (custom)
├── daily_revenue table
├── dim_customers table
├── fact_orders table
├── customer_status_snapshot merge (timestamp)
├── hourly_order_activity delete_insert (timestamp, microbatch)
├── daily_activity_rollup delete_insert (timestamp, microbatch)
├── hourly_activity_with_daily_context delete_insert (timestamp, microbatch)
└── order_status_index delete_insert (integer)

Functions (2)
├── table_fn__customer_orders table function
└── udf__is_completed_order sql udf

Seeds (1)
waffle_types
└── waffle_types (current)

✓ Plan complete 13 selected
✓ Plan complete 15 selected, 2 sources to load
```

All models now show `Normal` instead of `First run`.
The models and functions are no longer listed as first run, and the seed is current. The two loader-managed sources load on every build.

## 5. Common operations

Expand Down
Loading