Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 38 additions & 13 deletions src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Online: https://sqlbuild.com/docs/cli/build/
## Contents

- Usage
- Global options
- Flags
- Fast iteration
- Execution order
Expand All @@ -27,6 +28,18 @@ Compiles, plans, and executes the selected build lifecycle, running the full sel
sqb --project-dir <path> build [flags]
```

## Global options

These options apply to every `sqb` command and go before the command name, as in
`sqb --debug build`:

| Option | Description |
|--------|-------------|
| `--project-dir <path>` | Run against the project in `<path>` instead of the current directory |
| `--no-color` | Print without colour |
| `--debug` | Print detailed progress, the SQL each step runs, and internal diagnostics to stderr; implies `--verbose` |
| `--version`, `-V` | Print the SQLBuild version |

## Flags

| Flag | Description |
Expand All @@ -50,7 +63,7 @@ sqb --project-dir <path> build [flags]
| `--load` | Explicitly load managed sources before building |
| `--no-load` | Skip automatic source loading |
| `--reload` | Reload managed sources (passes `is_reload=True` to loaders) |
| `--include-stale-upstreams` | Expand selection to include stale upstream models needed for coherence |
| `--selection-diagnostics` | Warn when a selected model will build on changed upstream models that are not selected; see [Selection and staleness](../concepts/planning/selection-and-staleness.md) |
| `--manifest` | Generate `target/manifest.json` with plan-aware project metadata |
| `--select`, `-s` | Select specific models |
| `--exclude` | Exclude specific models |
Expand All @@ -69,7 +82,7 @@ This replaces the former `sqb run` command. The full lifecycle (tests + audits)
## Execution order

1. Managed sources are loaded (unless `--no-load`)
2. Seeds are loaded (if changed)
2. Selected seeds are loaded
3. Source audits run before their dependent models (unless `--no-audits`)
4. SQL unit tests run before their target model (unless `--no-tests`)
5. Models are materialized in DAG topological order
Expand All @@ -84,24 +97,36 @@ audit reads, so the audit can gate the resource like its other audits. See
```
Execution sqb build (concurrency: 1)

1/13 seed waffle_types OK 0.09s
2/13 view stg_customers OK 0.05s
audit not_null (customer_id) PASS
audit unique (customer_id) PASS
3/13 view stg_orders OK 0.03s
1/17 source raw__customers OK 0.05s rows=5
2/17 source raw__orders OK 0.03s rows=10

3/17 seed waffle_types OK 0.03s
7/17 view stg_orders OK 0.04s
test test_stg_orders PASS
expect expected stg_orders PASS
expect assertion order_ids_are_not_null PASS
audit not_null (order_id) PASS
audit unique (order_id) PASS
10/13 table hourly_order_activity (delete_insert) OK 0.16s
audit (d) expression_is_true PASS 4/4
audit (d) not_null (activity_hour) PASS 4/4
audit (f) expression_is_true PASS
audit not_null (customer_id) PASS
audit relationships (waffle_type_id) PASS
audit accepted_values (status) PASS
13/17 table hourly_order_activity (delete_insert) OK 0.24s
5 batches (1d) range 2026-03-31T09:00:00 → 2026-04-04T14:00:00 8 rows
audit (d) orders_placed_is_non_negative PASS 5/5
audit (d) not_null (activity_hour) PASS 5/5
audit (f) orders_placed_is_non_negative PASS
audit (f) not_null (activity_hour) PASS

Completed successfully.
PASS=66 WARN=0 FAIL=0 SKIP=0 TOTAL=66 (1.09s)
✓ Completed successfully PASS=99 WARN=0 FAIL=0 INSUFFICIENT=0 SKIP=0 TOTAL=99 (2.13s)
```

Each resource prints one row when it finishes, followed by its tests and audits. Rows for
individual warehouse statements and for steps inside a resource, such as `Create staging relation`,
`Promote relation`, `loader callable`, or audit evaluation, appear only with `sqb --debug build`.
Without `--debug`, a statement that is still running after 30 seconds prints a `RUNNING` row every
30 seconds, with its elapsed time and, when the warehouse reports one, its query ID. A failed
statement or step still prints its `FAIL` row.

## Deferred builds

Use `--defer-to` to resolve unselected model references against another target. This lets you build a subset of models in dev while referencing production tables for everything else:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ Copies selected relations from one target to another. It uses adapter-native clo

No `manifest.json` generation or artifact management is required. Clone works directly against live targets.

When `--to` is omitted, the destination is the active target selected by `--target`,
`sqlbuild_local.toml`, or `default_target`, in that order.
When `--to` is omitted, the destination is the active target selected by `sqlbuild_local.toml`
or `default_target`, in that order.

`--from` selects only the origin namespace and its clone-origin policy. `--to`, or the active
target when it is omitted, selects the sole physical connection used by the operation. The
Expand Down
20 changes: 10 additions & 10 deletions src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/compile.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ sqb --project-dir <path> compile [flags]
## What compile does

1. **Discovery** - finds `sqlbuild_project.toml`, scans for models, sources, seeds, functions, audits, tests, and macros
2. **Graph resolution** - resolves `ref()` and `source()` calls, expands macros, orders models by dependency
2. **Graph resolution** - resolves `__ref()`, `__source()`, and `__seed()` references, expands macros, orders models by dependency
3. **SQL validation** - validates SQL syntax (when SQL analysis is enabled)
4. **Column lineage** - analyzes column-level dependencies across models (fast mode by default)
5. **Contract validation** - checks declared column contracts against inferred query output
Expand Down Expand Up @@ -217,17 +217,17 @@ sqb compile
```

```
Compile ready (12 models)
Compile ready 12 models

stg_customers OK 3 columns
stg_orders OK 5 columns
stg_payments OK 4 columns
fact_orders OK 6 columns
dim_customers OK 4 columns
daily_revenue OK 3 columns
...
├── daily_revenue OK 6 columns
├── dim_customers OK 7 columns
├── fact_orders OK 13 columns
├── stg_customers OK 5 columns
├── stg_orders OK 6 columns
└── stg_payments OK 6 columns
...

Compiled: 12 models, 1 seed, 5 functions, 0 errors, 0 warnings
✓ Project compiled 12 models, 1 seed, 2 functions, 0 errors, 0 warnings
Wrote: target/compiled/
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ Each node represents a source, seed, model, or function:
| Field | Description |
|-------|-------------|
| `id` | Unique identifier (`{kind}:{name}`) |
| `kind` | `source`, `seed`, `model`, or `function` |
| `kind` | `source`, `seed`, `model`, `udf` (scalar function), or `table_fn` (table function) |
| `name` | Resource name |
| `asset_key` | Tuple used as the Dagster asset key (typically `[schema, name]` or `[database, schema, name]`) |
| `target` | Warehouse identity (database, schema, name, qualified_name) |
Expand Down
30 changes: 20 additions & 10 deletions src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/debug.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Online: https://sqlbuild.com/docs/cli/debug/

Checks that your project config is valid, the adapter is resolvable, and the warehouse connection works. Useful for diagnosing setup issues.

For detailed progress from any other command, use the global [`--debug`](build.md#global-options) option instead, as in `sqb --debug build`.

## Usage

```bash
Expand All @@ -16,11 +18,13 @@ sqb debug [flags]

## What it checks

The command runs three groups of checks:
The command reports four groups of checks:

**Runtime** - SQLBuild version, Python version, Python path, OS info.

**Configuration** - Finds and validates `sqlbuild_project.toml`, loads `sqlbuild_local.toml` if present, resolves the adapter and active target. For Snowflake it also shows the warehouse each [command group](../concepts/project-configuration.md#command-group-warehouses) connects with and where that value came from.
**Configuration** - Finds and validates `sqlbuild_project.toml`, loads `sqlbuild_local.toml` if present, resolves the adapter, active target, connection, database, and schema. For Snowflake it also shows the warehouse each [command group](../concepts/project-configuration.md#command-group-warehouses) connects with and where that value came from.

**Providers** - Lists the [providers](../concepts/python-nodes/providers.md) the project defines.

**Connection** - Displays connection settings as `sqb debug` itself uses them, with the `query` warehouse group applied (secrets are masked), attempts to connect to the warehouse, and runs `SELECT 1` to verify query execution.

Expand All @@ -42,7 +46,7 @@ sqb debug
SQLBuild Diagnostics

Runtime:
sqlbuild version: 0.2.1
sqlbuild version: 0.134.8
python version: 3.14.3
python path: /home/user/.venv/bin/python3
os info: Linux-6.6.87
Expand All @@ -53,18 +57,24 @@ Configuration:
project: waffle_shop [OK loaded]
adapter: snowflake [OK found]
target: dev [OK resolved]
connection: snowflake_dev [OK resolved]
database: ANALYTICS [OK resolved]
schema: DEV [OK resolved]
build warehouse: BUILD_WH [OK sqlbuild_project.toml [targets.dev.warehouses] build]
query warehouse: SQB_WH [OK connection warehouse]
query warehouse: TRANSFORM_WH [OK connection warehouse]

Providers:
providers: none discovered [OK none discovered]

Connection:
account: FJMQFQV-OJ66172
account: myorg-myaccount
authenticator: programmatic_access_token
database: SQB_DB
role: role_sqb_test
schema: TEST
database: ANALYTICS
role: TRANSFORMER
schema: DEV
token: ****
user: svc_sqb_test
warehouse: SQB_WH
user: SQLBUILD_USER
warehouse: TRANSFORM_WH
connection test: [OK connected]
query test: [OK SELECT 1]
```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,11 +66,11 @@ sqb freshness
Source freshness

Observed (3)
raw_customers timestamp 2026-06-05T14:30:00 adapter
raw_orders timestamp 2026-06-05T15:45:00 column tolerance 15m
raw_payments integer 42871 column
├── raw_customers value 2026-06-05T14:30:00+00:00 kind timestamp via column
├── raw_orders value 2026-06-05T12:00:00+00:00 kind timestamp via column tolerance 15m
└── raw_payments value 2026-06-05T14:28:00+00:00 kind timestamp via column tolerance 15m

Summary: observed=3 changed=0 unchanged=0 tolerated=0 unknown=0 errors=0
OBSERVED=3 CHANGED=0 UNCHANGED=0 TOLERATED=0 UNKNOWN=0 ERROR=0
```

Sources without explicit `freshness:` config are auto-observed using the `adapter` strategy if the adapter supports table metadata. Sources that can't be observed (expression sources, managed sources without freshness config on unsupported adapters) show as `unknown`.
Expand All @@ -93,15 +93,15 @@ sqb freshness --state
Source freshness

Changed (1)
raw_orders previous 2026-06-05T12:00:00 current 2026-06-05T15:45:00 tolerance 15m
└── raw_orders previous 2026-06-05T12:00:00+00:00 current 2026-06-05T15:45:00+00:00 tolerance 15m

Unchanged (1)
raw_customers previous 2026-06-05T14:30:00 current 2026-06-05T14:30:00
└── raw_customers previous 2026-06-05T14:30:00+00:00 current 2026-06-05T14:30:00+00:00

Tolerated (1)
raw_payments previous 2026-06-05T14:28:00 current 2026-06-05T14:30:00 tolerance 15m
└── raw_payments previous 2026-06-05T14:28:00+00:00 current 2026-06-05T14:30:00+00:00 tolerance 15m

Summary: observed=0 changed=1 unchanged=1 tolerated=1 unknown=0 errors=0
OBSERVED=0 CHANGED=1 UNCHANGED=1 TOLERATED=1 UNKNOWN=0 ERROR=0
```

### Statuses
Expand Down Expand Up @@ -145,15 +145,16 @@ sqb freshness --json
"status": "observed",
"strategy": "column",
"value_kind": "timestamp",
"current_data_version": "2026-06-05T15:45:00",
"current_data_version": "2026-06-05T15:45:00+00:00",
"previous_data_version": null,
"lag_tolerance": "15m",
"target": {
"database": null,
"schema": "raw",
"name": "orders"
},
"message": null
"message": null,
"age_status": null
}
],
"summary": {
Expand All @@ -162,7 +163,11 @@ sqb freshness --json
"unchanged": 0,
"tolerated": 0,
"unknown": 0,
"errors": 0
"errors": 0,
"age_pass": 0,
"age_warn": 0,
"age_error": 0,
"age_unknown": 0
}
}
```
Expand Down
28 changes: 19 additions & 9 deletions src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/init.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,14 @@ No flags. Run in the directory where you want to create the project.

## Project layout

`sqb init` creates the configuration, linter settings, and empty resource directories needed for a standalone project:
`sqb init` creates the configuration, empty resource directories needed for a standalone project,
and the SQLBuild agent skill (see [`sqb skills`](skills.md)):

```text
my-project/
sqlbuild_project.toml
.sqruff
.agents/skills/sqlbuild/
.claude/skills/sqlbuild/
models/
staging/
marts/
Expand Down Expand Up @@ -55,23 +57,31 @@ These top-level roles are project-wide. Once an audit, schema, hook, or macro is
one folder, such as `models/marts/`, SQLBuild requires it to move into that folder's `_sqlbuild/`
directory; see [Where to Put Declarations](../concepts/declaration-scopes/placement.md).

The generated project uses DuckDB, creates a named `local` connection shared by `dev` and
`prod`, and defaults models to table materialization. Its configuration follows this shape:
The generated project uses DuckDB, creates a named `developer` connection shared by `dev` and
`prod`, defaults models to table materialization, and makes audits warn by default. Its
configuration follows this shape:

```toml
name = "my_project"
adapter = "duckdb"
default_target = "dev"

[connections.local]
[connections.developer]
database = "my_project.duckdb"

[targets.dev]
connection = "local"
schema = "dev"
[settings]
default_audit_severity = "warn"

[defaults]
materialized = "table"

[targets.prod]
connection = "local"
connection = "developer"
schema = "prod"

[targets.dev]
connection = "developer"
schema = "dev"
```

The project name is derived from the current directory name, with hyphens converted to
Expand Down
27 changes: 14 additions & 13 deletions src/sqlbuild/.agents/skills/sqlbuild/references/docs/cli/lineage.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,15 +66,15 @@ upstream
├── model stg_payments models/staging/stg_payments.sql
│ └── source raw__payments sources/raw.yml
├── seed waffle_types seeds/waffle_types.csv
└── function udf__is_completed_order functions/sql/udf__is_completed_order.sql
└── udf udf__is_completed_order
downstream
├── model customer_status_snapshot models/intermediate/customer_status_snapshot.sql
├── model hourly_order_activity models/marts/hourly_order_activity.sql
│ ├── model daily_activity_rollup models/marts/daily_activity_rollup.sql
│ │ └── model hourly_activity_with_daily_context models/marts/hourly_activity_with_daily_context.sql
│ │ └── model hourly_order_activity (already shown)
│ └── model hourly_activity_with_daily_context (already shown)
└── model order_status_index models/intermediate/order_status_index.sql
│ └── model hourly_activity_with_daily_context models/marts/hourly_activity_with_daily_context.sql (already shown)
├── model order_status_index models/intermediate/order_status_index.sql
└── table_fn table_fn__customer_orders
```

Cycles and repeated nodes are annotated with "(already shown)" to avoid infinite recursion.
Expand All @@ -88,11 +88,12 @@ sqb lineage fact_orders --format list
```

```
source:raw__orders -> model:stg_orders
source:raw__payments -> model:stg_payments
model:stg_orders -> model:fact_orders
model:stg_payments -> model:fact_orders
seed:waffle_types -> model:fact_orders
udf:udf__is_completed_order -> model:fact_orders
model:stg_orders -> model:fact_orders
seed:waffle_types -> model:fact_orders
model:stg_payments -> model:fact_orders
source:raw__orders -> model:stg_orders
source:raw__payments -> model:stg_payments
```

### JSON
Expand Down Expand Up @@ -152,8 +153,8 @@ sqb lineage fact_orders.total_cents
```
Column trace fact_orders.total_cents upstream

<- stg_payments.amount_cents (expression)
<- raw__payments.amount_cents (direct)
└── stg_payments.amount_cents (expression)
└── raw__payments.amount_cents (direct)
```

### List
Expand All @@ -165,8 +166,8 @@ sqb lineage fact_orders.total_cents --format list
```
Column dependencies

stg_payments.amount_cents -> fact_orders.total_cents expression
raw__payments.amount_cents -> stg_payments.amount_cents direct
stg_payments.amount_cents -> fact_orders.total_cents expression
raw__payments.amount_cents -> stg_payments.amount_cents direct
```

### JSON
Expand Down
Loading
Loading