Skip to content

Commit b274479

Browse files
authored
docs: organise README demos into an In the terminal section (#843)
1 parent 170ef72 commit b274479

12 files changed

Lines changed: 123 additions & 43 deletions

File tree

‎.github/demos/contract.gif‎

174 KB
Loading

‎.github/demos/rename.gif‎

-17.3 KB
Loading

‎.github/demos/rules.gif‎

131 KB
Loading

‎.github/demos/scope.gif‎

-10.4 KB
Loading

‎README.md‎

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,9 @@ manifest files and no paid tier.
2222
<img src="https://raw.githubusercontent.com/chio-labs/sqlbuild/main/.github/demos/rename.gif" alt="Moving and renaming an incremental model: sqb plan migrates the existing table instead of rebuilding it" width="100%">
2323
</p>
2424

25+
Move an incremental model to a new folder and name, and `sqb plan` migrates its table instead of
26+
rebuilding it.
27+
2528
## Quick start
2629

2730
```bash
@@ -71,12 +74,47 @@ The playground runs on local DuckDB, with no warehouse credentials.
7174
- **Tidy up safely.** The [janitor](https://sqlbuild.com/docs/cli/janitor/) archives stale tables
7275
before anything is deleted.
7376

77+
Ingestion with Python loaders, and Python tasks, assets and checks, run in the same graph as your SQL
78+
models. See the [docs](https://sqlbuild.com/docs/) for everything else.
79+
80+
## In the terminal
81+
82+
Each demo runs on local DuckDB with the projects in [`website/examples`](website/examples). The
83+
tapes that record them are in [`website/recordings`](website/recordings).
84+
85+
### A renamed column breaks the contract
86+
87+
`daily_revenue` declares `contract enforced`. Renaming `waffles_sold` to `units_sold` in the
88+
`SELECT` fails at compile time, before anything reaches the warehouse.
89+
7490
<p align="center">
75-
<img src="https://raw.githubusercontent.com/chio-labs/sqlbuild/main/.github/demos/scope.gif" alt="sqb scope previews a move: the enum and macro the model uses would be lost, so the move is flagged before anything breaks" width="100%">
91+
<img src="https://raw.githubusercontent.com/chio-labs/sqlbuild/main/.github/demos/contract.gif" alt="sqb compile fails because units_sold is not in the enforced contract and the declared waffles_sold column is missing" width="100%">
7692
</p>
7793

78-
Ingestion with Python loaders, and Python tasks, assets and checks, run in the same graph as your SQL
79-
models. See the [docs](https://sqlbuild.com/docs/) for everything else.
94+
Both sides are reported: the new column isn't in the contract, and the declared one is gone.
95+
96+
### Your own conventions as compile errors
97+
98+
The project's rule says marts must read sources through staging. A mart that reads
99+
`__source("raw__payments")` directly fails, and so does the unit test that has no mock for it.
100+
101+
<p align="center">
102+
<img src="https://raw.githubusercontent.com/chio-labs/sqlbuild/main/.github/demos/rules.gif" alt="sqb compile reports the custom rule XSQBRARCH001 on the line that reads a raw source, and a unit test with no mock for that source" width="100%">
103+
</p>
104+
105+
The rule is [a short Python function](website/examples/waffle-shop/rules/layers.py). See
106+
[rules](https://sqlbuild.com/docs/concepts/rules/).
107+
108+
### See what a move would break
109+
110+
`sqb scope --as-path` previews moving a model before you move it.
111+
112+
<p align="center">
113+
<img src="https://raw.githubusercontent.com/chio-labs/sqlbuild/main/.github/demos/scope.gif" alt="sqb scope previews moving daily_revenue: the enum and macro it uses would be lost, so both usages are invalidated" width="100%">
114+
</p>
115+
116+
Look at `Lost` and `Invalidated usages`: the model uses an enum and a macro that are private to
117+
`models/marts`, so moving it would break both.
80118

81119
## Example
82120

‎website/recordings/README.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,18 +7,21 @@ scripts are the source; the rendered GIF, MP4 and PNG files go to `out/`, which
77
| Demo | Shows |
88
|------|-------|
99
| `rename` | A model is moved and renamed, and `sqb plan` migrates the existing table. |
10-
| `compile` | `sqb compile` catches an unknown column and a type mismatch. |
10+
| `contract` | A renamed column fails `sqb compile` against an enforced contract. |
11+
| `rules` | A custom rule and a missing test mock fail `sqb compile`. |
1112
| `scope` | `sqb scope --as-path` previews what a move would lose. |
1213
| `janitor` | `sqb janitor` archives a stale table before anything is deleted. |
13-
| `quickstart` | The playground: `sqb playground`, `sqb build` and `sqb test`. |
14+
| `compile` | `sqb compile` catches an unknown column and a type mismatch. Not used yet: the column checks it shows are unreleased. |
1415

15-
The README uses `rename.gif` and `scope.gif`, copied into `.github/demos/`. Each demo runs locally on DuckDB, using the projects in [`../examples`](../examples).
16+
The README uses `rename`, `contract`, `rules` and `scope`, copied into `.github/demos/`. Each demo
17+
runs locally on DuckDB, using the projects in [`../examples`](../examples).
1618

1719
## Requirements
1820

1921
- `vhs`, `ttyd` and `ffmpeg` on `PATH`, plus a Chromium or Chrome that VHS can find (it downloads
2022
one if none is on `PATH`).
2123
- DejaVu Sans Mono, the font the tapes use.
24+
- `tree` (the rename and scope tapes show folder trees).
2225
- `sqb` on `PATH`, or `SQB_BIN` pointing at a directory that contains the `sqb` to record.
2326

2427
## Record

‎website/recordings/prepare.sh‎

Lines changed: 29 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,16 +8,41 @@ dir="$here/scratch/$demo"
88
rm -rf "$dir"
99
mkdir -p "$here/scratch"
1010
case "$demo" in
11-
quickstart)
12-
mkdir -p "$dir" ;;
1311
rename)
14-
# Build the model under its old name, so the tape can move and rename it.
12+
# Build the model under its old name, so the tape can move it back to models/finance.
1513
cp -r "$examples/hero-shop" "$dir"
1614
cd "$dir"
1715
mkdir -p models/marts
1816
mv models/finance/daily_revenue.sql models/marts/revenue.sql
19-
rmdir models/finance
2017
sqb build >/dev/null ;;
18+
contract)
19+
# Give daily_revenue an enforced contract, then rename one of its columns in the SELECT.
20+
cp -r "$examples/waffle-shop" "$dir"
21+
cd "$dir"
22+
python3 - <<'PY'
23+
path = "models/marts/daily_revenue.sql"
24+
text = open(path).read()
25+
text = text.replace(" materialized table,\n", " materialized table,\n contract enforced,\n", 1)
26+
text = text.replace(
27+
" revenue_date (nullable false),\n",
28+
" revenue_date (type DATE, nullable false),\n"
29+
' order_count (description "Orders placed that day"),\n'
30+
' waffles_sold (description "Waffles sold that day"),\n'
31+
' total_revenue_cents (description "Successful payments in cents"),\n'
32+
' total_revenue_dollars (description "Successful payments in dollars"),\n'
33+
' avg_order_value_cents (description "Average order value in cents"),\n',
34+
1,
35+
)
36+
text = text.replace("AS waffles_sold", "AS units_sold", 1)
37+
open(path, "w").write(text)
38+
PY
39+
;;
40+
rules)
41+
# Make a mart read a raw source directly, which the project's layering rule forbids.
42+
cp -r "$examples/waffle-shop" "$dir"
43+
cd "$dir"
44+
sed -i 's/LEFT JOIN __ref("stg_payments") p/LEFT JOIN __source("raw__payments") p/' \
45+
models/marts/fact_orders.sql ;;
2146
scope)
2247
cp -r "$examples/waffle-shop" "$dir"
2348
cd "$dir"
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
Source tapes/common.tape
2+
Set FontSize 16
3+
Set Width 1600
4+
Set Height 820
5+
Output out/contract.gif
6+
Output out/contract.mp4
7+
Hide
8+
Type "cd scratch/contract && export PS1='$ ' && clear"
9+
Enter
10+
Sleep 500ms
11+
Show
12+
Type "sqb compile --select daily_revenue"
13+
Sleep 300ms
14+
Enter
15+
Wait@60s /\$\s*$/
16+
Sleep 1s
17+
Screenshot out/contract.png
18+
Sleep 15s

‎website/recordings/tapes/quickstart.tape‎

Lines changed: 0 additions & 25 deletions
This file was deleted.
Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,23 @@
11
Source tapes/common.tape
2+
Set Height 800
23
Output out/rename.gif
34
Output out/rename.mp4
45
Hide
56
Type "cd scratch/rename && export PS1='$ ' && clear"
67
Enter
78
Sleep 500ms
89
Show
9-
Type "find models -name '*.sql'"
10+
Type "tree --noreport models"
1011
Enter
1112
Sleep 1500ms
12-
Type "mkdir models/finance && mv models/marts/revenue.sql models/finance/daily_revenue.sql"
13+
Type "mv models/marts/revenue.sql models/finance/daily_revenue.sql"
1314
Sleep 300ms
1415
Enter
1516
Sleep 800ms
1617
Type "sqb plan"
1718
Sleep 300ms
1819
Enter
19-
Sleep 5s
20+
Wait@60s /\$\s*$/
21+
Sleep 1s
2022
Screenshot out/rename.png
21-
Sleep 3s
23+
Sleep 10s

0 commit comments

Comments
 (0)