Skip to content

Commit 2979009

Browse files
authored
docs: correct out-of-date and wrong documentation (#977)
* docs: document --debug and correct test-only macro placement Add global options, including --debug, to the build reference and explain which progress rows need --debug. Describe where macros used only by tests can live. Fix outdated flags, sample output and file layouts across CLI and concept pages. * docs: correct audit defaults, seed loading and selector behaviour Audits default to error severity and the delta_and_final run scope, seeds load on every build, path-between selectors include every path, and project concurrency is no longer serial-only. Refresh selector error samples. * docs: fix custom adapter imports and dbt bootstrap sample Custom adapter examples now import from the real module paths and call execute with keyword arguments. The generated dbt connection is named <target>_dbt_profile. The quickstart lists the loader and generic audit where the playground puts them. * docs: refresh the P006 macro reference sample * docs: regenerate the SQLBuild skill references
1 parent 03d2922 commit 2979009

47 files changed

Lines changed: 578 additions & 350 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/build.md‎

Lines changed: 38 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Online: https://sqlbuild.com/docs/cli/build/
99
## Contents
1010

1111
- Usage
12+
- Global options
1213
- Flags
1314
- Fast iteration
1415
- Execution order
@@ -27,6 +28,18 @@ Compiles, plans, and executes the selected build lifecycle, running the full sel
2728
sqb --project-dir <path> build [flags]
2829
```
2930

31+
## Global options
32+
33+
These options apply to every `sqb` command and go before the command name, as in
34+
`sqb --debug build`:
35+
36+
| Option | Description |
37+
|--------|-------------|
38+
| `--project-dir <path>` | Run against the project in `<path>` instead of the current directory |
39+
| `--no-color` | Print without colour |
40+
| `--debug` | Print detailed progress, the SQL each step runs, and internal diagnostics to stderr; implies `--verbose` |
41+
| `--version`, `-V` | Print the SQLBuild version |
42+
3043
## Flags
3144

3245
| Flag | Description |
@@ -50,7 +63,7 @@ sqb --project-dir <path> build [flags]
5063
| `--load` | Explicitly load managed sources before building |
5164
| `--no-load` | Skip automatic source loading |
5265
| `--reload` | Reload managed sources (passes `is_reload=True` to loaders) |
53-
| `--include-stale-upstreams` | Expand selection to include stale upstream models needed for coherence |
66+
| `--selection-diagnostics` | Warn when a selected model will build on changed upstream models that are not selected; see [Selection and staleness](../concepts/planning/selection-and-staleness.md) |
5467
| `--manifest` | Generate `target/manifest.json` with plan-aware project metadata |
5568
| `--select`, `-s` | Select specific models |
5669
| `--exclude` | Exclude specific models |
@@ -69,7 +82,7 @@ This replaces the former `sqb run` command. The full lifecycle (tests + audits)
6982
## Execution order
7083

7184
1. Managed sources are loaded (unless `--no-load`)
72-
2. Seeds are loaded (if changed)
85+
2. Selected seeds are loaded
7386
3. Source audits run before their dependent models (unless `--no-audits`)
7487
4. SQL unit tests run before their target model (unless `--no-tests`)
7588
5. Models are materialized in DAG topological order
@@ -84,24 +97,36 @@ audit reads, so the audit can gate the resource like its other audits. See
8497
```
8598
Execution sqb build (concurrency: 1)
8699
87-
1/13 seed waffle_types OK 0.09s
88-
2/13 view stg_customers OK 0.05s
89-
audit not_null (customer_id) PASS
90-
audit unique (customer_id) PASS
91-
3/13 view stg_orders OK 0.03s
100+
1/17 source raw__customers OK 0.05s rows=5
101+
2/17 source raw__orders OK 0.03s rows=10
102+
103+
3/17 seed waffle_types OK 0.03s
104+
7/17 view stg_orders OK 0.04s
92105
test test_stg_orders PASS
106+
expect expected stg_orders PASS
107+
expect assertion order_ids_are_not_null PASS
93108
audit not_null (order_id) PASS
94109
audit unique (order_id) PASS
95-
10/13 table hourly_order_activity (delete_insert) OK 0.16s
96-
audit (d) expression_is_true PASS 4/4
97-
audit (d) not_null (activity_hour) PASS 4/4
98-
audit (f) expression_is_true PASS
110+
audit not_null (customer_id) PASS
111+
audit relationships (waffle_type_id) PASS
112+
audit accepted_values (status) PASS
113+
13/17 table hourly_order_activity (delete_insert) OK 0.24s
114+
5 batches (1d) range 2026-03-31T09:00:00 → 2026-04-04T14:00:00 8 rows
115+
audit (d) orders_placed_is_non_negative PASS 5/5
116+
audit (d) not_null (activity_hour) PASS 5/5
117+
audit (f) orders_placed_is_non_negative PASS
99118
audit (f) not_null (activity_hour) PASS
100119
101-
Completed successfully.
102-
PASS=66 WARN=0 FAIL=0 SKIP=0 TOTAL=66 (1.09s)
120+
✓ Completed successfully PASS=99 WARN=0 FAIL=0 INSUFFICIENT=0 SKIP=0 TOTAL=99 (2.13s)
103121
```
104122

123+
Each resource prints one row when it finishes, followed by its tests and audits. Rows for
124+
individual warehouse statements and for steps inside a resource, such as `Create staging relation`,
125+
`Promote relation`, `loader callable`, or audit evaluation, appear only with `sqb --debug build`.
126+
Without `--debug`, a statement that is still running after 30 seconds prints a `RUNNING` row every
127+
30 seconds, with its elapsed time and, when the warehouse reports one, its query ID. A failed
128+
statement or step still prints its `FAIL` row.
129+
105130
## Deferred builds
106131

107132
Use `--defer-to` to resolve unselected model references against another target. This lets you build a subset of models in dev while referencing production tables for everything else:

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/clone.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,8 @@ Copies selected relations from one target to another. It uses adapter-native clo
1010

1111
No `manifest.json` generation or artifact management is required. Clone works directly against live targets.
1212

13-
When `--to` is omitted, the destination is the active target selected by `--target`,
14-
`sqlbuild_local.toml`, or `default_target`, in that order.
13+
When `--to` is omitted, the destination is the active target selected by `sqlbuild_local.toml`
14+
or `default_target`, in that order.
1515

1616
`--from` selects only the origin namespace and its clone-origin policy. `--to`, or the active
1717
target when it is omitted, selects the sole physical connection used by the operation. The

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/compile.md‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ sqb --project-dir <path> compile [flags]
4343
## What compile does
4444

4545
1. **Discovery** - finds `sqlbuild_project.toml`, scans for models, sources, seeds, functions, audits, tests, and macros
46-
2. **Graph resolution** - resolves `ref()` and `source()` calls, expands macros, orders models by dependency
46+
2. **Graph resolution** - resolves `__ref()`, `__source()`, and `__seed()` references, expands macros, orders models by dependency
4747
3. **SQL validation** - validates SQL syntax (when SQL analysis is enabled)
4848
4. **Column lineage** - analyzes column-level dependencies across models (fast mode by default)
4949
5. **Contract validation** - checks declared column contracts against inferred query output
@@ -217,17 +217,17 @@ sqb compile
217217
```
218218

219219
```
220-
Compile ready (12 models)
220+
Compile ready 12 models
221221
222-
stg_customers OK 3 columns
223-
stg_orders OK 5 columns
224-
stg_payments OK 4 columns
225-
fact_orders OK 6 columns
226-
dim_customers OK 4 columns
227-
daily_revenue OK 3 columns
228-
...
222+
├── daily_revenue OK 6 columns
223+
├── dim_customers OK 7 columns
224+
├── fact_orders OK 13 columns
225+
├── stg_customers OK 5 columns
226+
├── stg_orders OK 6 columns
227+
└── stg_payments OK 6 columns
228+
...
229229
230-
Compiled: 12 models, 1 seed, 5 functions, 0 errors, 0 warnings
230+
✓ Project compiled 12 models, 1 seed, 2 functions, 0 errors, 0 warnings
231231
Wrote: target/compiled/
232232
```
233233

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/dag.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ Each node represents a source, seed, model, or function:
9696
| Field | Description |
9797
|-------|-------------|
9898
| `id` | Unique identifier (`{kind}:{name}`) |
99-
| `kind` | `source`, `seed`, `model`, or `function` |
99+
| `kind` | `source`, `seed`, `model`, `udf` (scalar function), or `table_fn` (table function) |
100100
| `name` | Resource name |
101101
| `asset_key` | Tuple used as the Dagster asset key (typically `[schema, name]` or `[database, schema, name]`) |
102102
| `target` | Warehouse identity (database, schema, name, qualified_name) |

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/debug.md‎

Lines changed: 20 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ Online: https://sqlbuild.com/docs/cli/debug/
88

99
Checks that your project config is valid, the adapter is resolvable, and the warehouse connection works. Useful for diagnosing setup issues.
1010

11+
For detailed progress from any other command, use the global [`--debug`](build.md#global-options) option instead, as in `sqb --debug build`.
12+
1113
## Usage
1214

1315
```bash
@@ -16,11 +18,13 @@ sqb debug [flags]
1618

1719
## What it checks
1820

19-
The command runs three groups of checks:
21+
The command reports four groups of checks:
2022

2123
**Runtime** - SQLBuild version, Python version, Python path, OS info.
2224

23-
**Configuration** - Finds and validates `sqlbuild_project.toml`, loads `sqlbuild_local.toml` if present, resolves the adapter and active target. For Snowflake it also shows the warehouse each [command group](../concepts/project-configuration.md#command-group-warehouses) connects with and where that value came from.
25+
**Configuration** - Finds and validates `sqlbuild_project.toml`, loads `sqlbuild_local.toml` if present, resolves the adapter, active target, connection, database, and schema. For Snowflake it also shows the warehouse each [command group](../concepts/project-configuration.md#command-group-warehouses) connects with and where that value came from.
26+
27+
**Providers** - Lists the [providers](../concepts/python-nodes/providers.md) the project defines.
2428

2529
**Connection** - Displays connection settings as `sqb debug` itself uses them, with the `query` warehouse group applied (secrets are masked), attempts to connect to the warehouse, and runs `SELECT 1` to verify query execution.
2630

@@ -42,7 +46,7 @@ sqb debug
4246
SQLBuild Diagnostics
4347
4448
Runtime:
45-
sqlbuild version: 0.2.1
49+
sqlbuild version: 0.134.8
4650
python version: 3.14.3
4751
python path: /home/user/.venv/bin/python3
4852
os info: Linux-6.6.87
@@ -53,18 +57,24 @@ Configuration:
5357
project: waffle_shop [OK loaded]
5458
adapter: snowflake [OK found]
5559
target: dev [OK resolved]
60+
connection: snowflake_dev [OK resolved]
61+
database: ANALYTICS [OK resolved]
62+
schema: DEV [OK resolved]
5663
build warehouse: BUILD_WH [OK sqlbuild_project.toml [targets.dev.warehouses] build]
57-
query warehouse: SQB_WH [OK connection warehouse]
64+
query warehouse: TRANSFORM_WH [OK connection warehouse]
65+
66+
Providers:
67+
providers: none discovered [OK none discovered]
5868
5969
Connection:
60-
account: FJMQFQV-OJ66172
70+
account: myorg-myaccount
6171
authenticator: programmatic_access_token
62-
database: SQB_DB
63-
role: role_sqb_test
64-
schema: TEST
72+
database: ANALYTICS
73+
role: TRANSFORMER
74+
schema: DEV
6575
token: ****
66-
user: svc_sqb_test
67-
warehouse: SQB_WH
76+
user: SQLBUILD_USER
77+
warehouse: TRANSFORM_WH
6878
connection test: [OK connected]
6979
query test: [OK SELECT 1]
7080
```

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/freshness.md‎

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -66,11 +66,11 @@ sqb freshness
6666
Source freshness
6767
6868
Observed (3)
69-
raw_customers timestamp 2026-06-05T14:30:00 adapter
70-
raw_orders timestamp 2026-06-05T15:45:00 column tolerance 15m
71-
raw_payments integer 42871 column
69+
├── raw_customers value 2026-06-05T14:30:00+00:00 kind timestamp via column
70+
├── raw_orders value 2026-06-05T12:00:00+00:00 kind timestamp via column tolerance 15m
71+
└── raw_payments value 2026-06-05T14:28:00+00:00 kind timestamp via column tolerance 15m
7272
73-
Summary: observed=3 changed=0 unchanged=0 tolerated=0 unknown=0 errors=0
73+
OBSERVED=3 CHANGED=0 UNCHANGED=0 TOLERATED=0 UNKNOWN=0 ERROR=0
7474
```
7575

7676
Sources without explicit `freshness:` config are auto-observed using the `adapter` strategy if the adapter supports table metadata. Sources that can't be observed (expression sources, managed sources without freshness config on unsupported adapters) show as `unknown`.
@@ -93,15 +93,15 @@ sqb freshness --state
9393
Source freshness
9494
9595
Changed (1)
96-
raw_orders previous 2026-06-05T12:00:00 current 2026-06-05T15:45:00 tolerance 15m
96+
└── raw_orders previous 2026-06-05T12:00:00+00:00 current 2026-06-05T15:45:00+00:00 tolerance 15m
9797
9898
Unchanged (1)
99-
raw_customers previous 2026-06-05T14:30:00 current 2026-06-05T14:30:00
99+
└── raw_customers previous 2026-06-05T14:30:00+00:00 current 2026-06-05T14:30:00+00:00
100100
101101
Tolerated (1)
102-
raw_payments previous 2026-06-05T14:28:00 current 2026-06-05T14:30:00 tolerance 15m
102+
└── raw_payments previous 2026-06-05T14:28:00+00:00 current 2026-06-05T14:30:00+00:00 tolerance 15m
103103
104-
Summary: observed=0 changed=1 unchanged=1 tolerated=1 unknown=0 errors=0
104+
OBSERVED=0 CHANGED=1 UNCHANGED=1 TOLERATED=1 UNKNOWN=0 ERROR=0
105105
```
106106

107107
### Statuses
@@ -145,15 +145,16 @@ sqb freshness --json
145145
"status": "observed",
146146
"strategy": "column",
147147
"value_kind": "timestamp",
148-
"current_data_version": "2026-06-05T15:45:00",
148+
"current_data_version": "2026-06-05T15:45:00+00:00",
149149
"previous_data_version": null,
150150
"lag_tolerance": "15m",
151151
"target": {
152152
"database": null,
153153
"schema": "raw",
154154
"name": "orders"
155155
},
156-
"message": null
156+
"message": null,
157+
"age_status": null
157158
}
158159
],
159160
"summary": {
@@ -162,7 +163,11 @@ sqb freshness --json
162163
"unchanged": 0,
163164
"tolerated": 0,
164165
"unknown": 0,
165-
"errors": 0
166+
"errors": 0,
167+
"age_pass": 0,
168+
"age_warn": 0,
169+
"age_error": 0,
170+
"age_unknown": 0
166171
}
167172
}
168173
```

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/init.md‎

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,14 @@ No flags. Run in the directory where you want to create the project.
1818

1919
## Project layout
2020

21-
`sqb init` creates the configuration, linter settings, and empty resource directories needed for a standalone project:
21+
`sqb init` creates the configuration, empty resource directories needed for a standalone project,
22+
and the SQLBuild agent skill (see [`sqb skills`](skills.md)):
2223

2324
```text
2425
my-project/
2526
sqlbuild_project.toml
26-
.sqruff
27+
.agents/skills/sqlbuild/
28+
.claude/skills/sqlbuild/
2729
models/
2830
staging/
2931
marts/
@@ -55,23 +57,31 @@ These top-level roles are project-wide. Once an audit, schema, hook, or macro is
5557
one folder, such as `models/marts/`, SQLBuild requires it to move into that folder's `_sqlbuild/`
5658
directory; see [Where to Put Declarations](../concepts/declaration-scopes/placement.md).
5759

58-
The generated project uses DuckDB, creates a named `local` connection shared by `dev` and
59-
`prod`, and defaults models to table materialization. Its configuration follows this shape:
60+
The generated project uses DuckDB, creates a named `developer` connection shared by `dev` and
61+
`prod`, defaults models to table materialization, and makes audits warn by default. Its
62+
configuration follows this shape:
6063

6164
```toml
65+
name = "my_project"
6266
adapter = "duckdb"
6367
default_target = "dev"
6468

65-
[connections.local]
69+
[connections.developer]
6670
database = "my_project.duckdb"
6771

68-
[targets.dev]
69-
connection = "local"
70-
schema = "dev"
72+
[settings]
73+
default_audit_severity = "warn"
74+
75+
[defaults]
76+
materialized = "table"
7177

7278
[targets.prod]
73-
connection = "local"
79+
connection = "developer"
7480
schema = "prod"
81+
82+
[targets.dev]
83+
connection = "developer"
84+
schema = "dev"
7585
```
7686

7787
The project name is derived from the current directory name, with hyphens converted to

‎src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/lineage.md‎

Lines changed: 14 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -66,15 +66,15 @@ upstream
6666
├── model stg_payments models/staging/stg_payments.sql
6767
│ └── source raw__payments sources/raw.yml
6868
├── seed waffle_types seeds/waffle_types.csv
69-
└── function udf__is_completed_order functions/sql/udf__is_completed_order.sql
69+
└── udf udf__is_completed_order
7070
downstream
7171
├── model customer_status_snapshot models/intermediate/customer_status_snapshot.sql
7272
├── model hourly_order_activity models/marts/hourly_order_activity.sql
7373
│ ├── model daily_activity_rollup models/marts/daily_activity_rollup.sql
7474
│ │ └── model hourly_activity_with_daily_context models/marts/hourly_activity_with_daily_context.sql
75-
│ │ └── model hourly_order_activity (already shown)
76-
│ └── model hourly_activity_with_daily_context (already shown)
77-
└── model order_status_index models/intermediate/order_status_index.sql
75+
│ └── model hourly_activity_with_daily_context models/marts/hourly_activity_with_daily_context.sql (already shown)
76+
├── model order_status_index models/intermediate/order_status_index.sql
77+
└── table_fn table_fn__customer_orders
7878
```
7979

8080
Cycles and repeated nodes are annotated with "(already shown)" to avoid infinite recursion.
@@ -88,11 +88,12 @@ sqb lineage fact_orders --format list
8888
```
8989

9090
```
91-
source:raw__orders -> model:stg_orders
92-
source:raw__payments -> model:stg_payments
93-
model:stg_orders -> model:fact_orders
94-
model:stg_payments -> model:fact_orders
95-
seed:waffle_types -> model:fact_orders
91+
udf:udf__is_completed_order -> model:fact_orders
92+
model:stg_orders -> model:fact_orders
93+
seed:waffle_types -> model:fact_orders
94+
model:stg_payments -> model:fact_orders
95+
source:raw__orders -> model:stg_orders
96+
source:raw__payments -> model:stg_payments
9697
```
9798

9899
### JSON
@@ -152,8 +153,8 @@ sqb lineage fact_orders.total_cents
152153
```
153154
Column trace fact_orders.total_cents upstream
154155
155-
<- stg_payments.amount_cents (expression)
156-
<- raw__payments.amount_cents (direct)
156+
└── stg_payments.amount_cents (expression)
157+
└── raw__payments.amount_cents (direct)
157158
```
158159

159160
### List
@@ -165,8 +166,8 @@ sqb lineage fact_orders.total_cents --format list
165166
```
166167
Column dependencies
167168
168-
stg_payments.amount_cents -> fact_orders.total_cents expression
169-
raw__payments.amount_cents -> stg_payments.amount_cents direct
169+
stg_payments.amount_cents -> fact_orders.total_cents expression
170+
raw__payments.amount_cents -> stg_payments.amount_cents direct
170171
```
171172

172173
### JSON

0 commit comments

Comments
 (0)