Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion src/sqlbuild/.agents/skills/sqlbuild/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: sqlbuild
description: ALWAYS load this skill when doing ANY SQLBuild work - models, sources, seeds, macros, enums, constants, tests, audits, scenarios, incremental and microbatch models, configuration, CLI, adapters, or dbt interop. Covers the verify loop (compile, plan, build), comparing data or two queries with sqb diff, inspecting macro/enum/constant visibility with sqb scope, impact analysis with sqb lineage, renaming or moving models and columns with sqb rename and sqb mv, and where to find exact syntax.
description: ALWAYS load this skill when doing ANY SQLBuild work - models, sources, seeds, macros, enums, constants, tests, audits, scenarios, incremental and microbatch models, configuration, CLI, adapters, or dbt interop. Covers the verify loop (compile, plan, build), comparing data or two queries with sqb diff, inspecting macro/enum/constant visibility with sqb scope, impact analysis with sqb lineage, renaming or moving models and columns with sqb rename and sqb mv, investigating what past runs did (statements, SQL, audit results, errors, output), locally or across a team's shared history, with sqb recall, and where to find exact syntax.
---

# SQLBuild
Expand Down Expand Up @@ -70,6 +70,27 @@ JOIN __ref("stg_payments") p USING (order_id)
When something fails, run `sqb debug` first. It separates configuration, authentication and
connection problems from command-specific errors. `sqb debug --no-connection` checks config only.

To find out what an earlier command did, query its recorded history with `sqb recall` instead of
rerunning it or hunting for log files. Recall is on by default and records every command: its
statements and SQL, audit results, errors, timing and exact output. `sqb recall schema` lists the
views (`invocations`, `statements`, `audit_results`, `artifacts` and more); `sqb recall query
"SELECT ..."` runs SQL over them; `sqb recall output last --tail 50` prints what the latest command
printed. Add `--backend warehouse` to check live warehouse state with the same command.

Check `[recall] connection` in the project config to see which store you are reading:

- **Local (the default).** A SQLite file, `logs/recall.sqlite3`, holding this checkout's own runs.
Use it for "what did my last build do?" and to compare runs during your session.
- **Shared ClickHouse store.** One history for every machine that writes to it: teammates, CI, and
scheduled or orchestrated production runs. This is far more powerful: you can see what prod
actually ran and printed last night without access to the machine, find the first run where an
audit started failing or a statement slowed down across all runs, and compare a production run
with yours by `target`, `git_revision`, `sqlbuild_version` or `host_token`. Queries use
ClickHouse SQL, and `--json` reports `rows_read` and `bytes_read`; filter by time and keep a
`LIMIT` on large stores.

Read [references/docs/concepts/observability/recall.md](references/docs/concepts/observability/recall.md).

## Reach for these tools

These are SQLBuild's highest-leverage features and the ones most often missed. Use them whenever
Expand All @@ -86,6 +107,8 @@ the trigger applies.
| Find what a model depends on, or what breaks if it changes | `sqb lineage <model> --direction both` |
| Trace where a column comes from, or who consumes it before renaming or dropping it | `sqb lineage <model>.<column> --direction downstream` |
| See which intermediate models a SQL test will really execute | `sqb test <model> --inspect` |
| Find out what a past run did: which models ran, their SQL, audit results, errors and timing | `sqb recall query "SELECT ... FROM statements"` (views: `sqb recall schema`) |
| See exactly what an earlier command printed, including a scheduled run | `sqb recall output last --tail 50` or `sqb recall output <invocation_id>` |
| Inspect the full graph as data | `sqb dag --json`, `sqb plan --json`, `sqb compile --json` |
| Adopt an existing warehouse table's columns into a contract | `sqb contract generate --from <target> --select <model>` |
| Try syntax safely in a throwaway DuckDB project | `sqb playground /tmp/sqb-sandbox` |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -182,9 +182,14 @@ notice and records nothing.

## Sharing history in ClickHouse

Point `connection` at a ClickHouse connection to keep one history for a whole team. The connection
needs `adapter = "clickhouse"` when the project uses another adapter, and its `database` must
already exist; Recall never creates databases.
Point `connection` at a ClickHouse connection to keep one history for a whole team. Every machine
that writes to it, including CI and scheduled production runs, becomes queryable from anywhere:
see what last night's production build ran and printed without access to the machine, find the
first run where an audit started failing across everyone's runs, or compare a production run with
your own by `target`, `git_revision` or `sqlbuild_version`.

The connection needs `adapter = "clickhouse"` when the project uses another adapter, and its
`database` must already exist; Recall never creates databases.

```toml
[connections.history]
Expand Down
11 changes: 8 additions & 3 deletions website/src/content/docs/docs/concepts/observability/recall.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -166,9 +166,14 @@ notice and records nothing.

## Sharing history in ClickHouse

Point `connection` at a ClickHouse connection to keep one history for a whole team. The connection
needs `adapter = "clickhouse"` when the project uses another adapter, and its `database` must
already exist; Recall never creates databases.
Point `connection` at a ClickHouse connection to keep one history for a whole team. Every machine
that writes to it, including CI and scheduled production runs, becomes queryable from anywhere:
see what last night's production build ran and printed without access to the machine, find the
first run where an audit started failing across everyone's runs, or compare a production run with
your own by `target`, `git_revision` or `sqlbuild_version`.

The connection needs `adapter = "clickhouse"` when the project uses another adapter, and its
`database` must already exist; Recall never creates databases.

```toml
[connections.history]
Expand Down
Loading