After production changes
++ Every run is recorded: what ran, the SQL it executed, audit results and everything it printed. Agents and people ask in SQL, then check live state with the same command. +
+sqb recall query "SELECT invocation_started_at AS run, outcome, violation_count + FROM audit_results WHERE audit_name = 'revenue_is_non_negative' + ORDER BY run DESC LIMIT 3" +run | outcome | violation_count +2026-10-20T00:01 | warn | 1 +2026-10-20T00:00 | pass | 0 +2026-10-19T23:59 | pass | 0 + +sqb recall output last --tail 4 + daily_revenue + audit revenue_is_non_negative returned 1 row +✓ Completed with warnings PASS=89 WARN=1 FAIL=0 … + +sqb recall query --backend warehouse "SELECT r.revenue_date, r.total_revenue_cents, + p.payment_id, p.amount_cents FROM prod.daily_revenue r JOIN prod.stg_payments p + ON CAST(p.paid_at AS DATE) = r.revenue_date WHERE p.amount_cents < 0" +revenue_date | total_revenue_cents | payment_id | amount_cents +2026-04-03 | -1250 | 9 | -2200+
Real output from SQLBuild's example project, trimmed to the relevant lines. During a migration the old table is never modified, archives can be renamed back until they expire, and every move is recorded. + In step 09 the audit started warning on the latest run, the build still finished, and the bad + row is still in prod: a refund recorded as a negative payment. Three queries, no log digging. + History stays in a SQLite file in your project, or in ClickHouse for a team. No service, no + account, no paid tier. How migrations work · - How the janitor works + How the janitor works · + How Recall works
From 433e443b70ca9d2ef4c7ae139c0247cb2604162b Mon Sep 17 00:00:00 2001 From: Kevin Longe <273747162+kvlonge@users.noreply.github.com> Date: Tue, 20 Oct 2026 01:06:59 +0100 Subject: [PATCH 2/2] docs: show a Recall investigation and point readers to Recall across the docs --- .../docs/concepts/adapters/clickhouse.md | 9 +++ .../docs/concepts/adapters/sqlite.md | 7 +++ .../references/docs/concepts/observability.md | 18 +++++- .../observability/execution-history.md | 4 ++ .../docs/concepts/observability/recall.md | 59 +++++++++++++++++++ .../references/docs/feature-comparison.md | 6 +- .../skills/sqlbuild/references/docs/index.md | 10 +++- .../sqlbuild/references/docs/roadmap.md | 4 +- website/public/index.html | 4 +- .../docs/concepts/adapters/clickhouse.mdx | 8 +++ .../docs/docs/concepts/adapters/sqlite.mdx | 6 ++ .../docs/docs/concepts/observability.mdx | 18 +++++- .../observability/execution-history.mdx | 4 ++ .../docs/concepts/observability/recall.mdx | 58 ++++++++++++++++++ .../content/docs/docs/feature-comparison.mdx | 6 +- website/src/content/docs/docs/index.mdx | 10 +++- website/src/content/docs/docs/roadmap.mdx | 4 +- 17 files changed, 217 insertions(+), 18 deletions(-) diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/adapters/clickhouse.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/adapters/clickhouse.md index b8bdd8f42d..c3cf447db9 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/adapters/clickhouse.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/adapters/clickhouse.md @@ -19,6 +19,7 @@ Online: https://sqlbuild.com/docs/concepts/adapters/clickhouse/ - Settings SQLBuild pins - SQL differences to watch - Limits +- Recall store The ClickHouse adapter is in beta and supports a single server. Clusters (`ON CLUSTER`, replicated and distributed tables) are not supported yet. Please [report issues](https://github.com/chio-labs/sqlbuild/issues). @@ -187,3 +188,11 @@ ClickHouse returns a type's default value (`0`, the epoch, an empty string) from - Snapshots always use an unsorted `MergeTree` table. - Seeds load through the HTTP insert interface as CSV, so large seeds never hit the `max_query_size` limit. - Retention and time travel are not available. + +## Recall store + +A ClickHouse connection can also hold [Recall](../observability/recall.md)'s command +history for a whole team, whichever adapter the project builds with. Set +`[recall] connection` to it; Recall creates its tables on first use, keeps every output byte by +default, and needs only `SELECT` and `INSERT` once the tables exist. See +[Sharing history in ClickHouse](../observability/recall.md#sharing-history-in-clickhouse). diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/adapters/sqlite.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/adapters/sqlite.md index 701f486d34..fc912192a9 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/adapters/sqlite.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/adapters/sqlite.md @@ -15,6 +15,7 @@ Online: https://sqlbuild.com/docs/concepts/adapters/sqlite/ - Capabilities - Scenarios - Limits +- Recall store SQLite uses the `sqlite3` module that ships with Python. No extra installation is needed. SQLBuild requires SQLite 3.44 or newer; Python builds installed by `uv` bundle a recent version. @@ -130,3 +131,9 @@ as `datetime(ordered_at, 'start of day')`, have no DuckDB translation and fail l | SQLite older than 3.44 | `A701` | | Another process holds the write lock beyond `busy_timeout` | `A702` | | Concurrent microbatch writes: one file has one writer | refused at plan time | + +## Recall store + +[Recall](../observability/recall.md) keeps its local history in a SQLite file, +`logs/recall.sqlite3`, for every project, whichever adapter it builds with. Point +`[recall] connection` at a `sqlite` connection to put the file somewhere else. diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/observability.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/observability.md index 1129254621..04e47f9354 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/observability.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/observability.md @@ -13,14 +13,28 @@ whether work started, completed, failed, skipped, or retried. |--------|---------|-----------| | `LifecycleEvent` | Immutable invocation, run, resource-attempt, operation, statement, retry, and audit facts | Canonical lifecycle evidence once durably stored | | `DiagnosticLog` | Structured framework diagnostics | Explanatory only | -| `stdout.log` / `stderr.log` | Exact host-local process output | Troubleshooting transcript | -| [Recall](observability/recall.md) | Local queryable history of invocations, statements, SQL, and output | Troubleshooting history; best effort | +| [Recall](observability/recall.md) | Queryable history of invocations, statements, SQL, audit results, and output, locally or in ClickHouse | Troubleshooting history; best effort | +| `stdout.log` / `stderr.log` | Exact host-local process output (compute log files) | Troubleshooting transcript; also stored by Recall | | `CommandOutputRecord` | Bounded, ANSI-free remote stdout/stderr chunks | Potentially sensitive and lossy troubleshooting data | | Final JSON output | End-of-command aggregate | Result projection, not event history | Human-readable CLI output remains the normal operator interface. Structured records are a separate contract for integrations and durable consumers. +## Which to use + +| You want to | Use | +|-------------|-----| +| Find out what a past run did: what ran, its SQL, audit results, errors and output | [Recall](observability/recall.md) and [`sqb recall`](../cli/recall.md) | +| Send events or output to your own systems, such as a message queue or a monitoring tool | [Typed sinks](observability/sinks.md) | +| Store lifecycle events and query run state from your own Python application | [Execution history](observability/execution-history.md) | +| Read the result of the command you just ran | Its final `--json` output | + +Recall is on by default and covers investigation for people and agents. Sinks and execution history +are for building on SQLBuild's events elsewhere. The `stdout.log`, `stderr.log` and +`diagnostics.jsonl` compute log files under `logs/sqb recall query "SELECT invocation_started_at AS run, outcome, violation_count
- FROM audit_results WHERE audit_name = 'revenue_is_non_negative'
+sqb recall query "SELECT substr(invocation_started_at, 1, 16) AS run, outcome,
+ violation_count FROM audit_results WHERE audit_name = 'revenue_is_non_negative'
ORDER BY run DESC LIMIT 3"
run | outcome | violation_count
2026-10-20T00:01 | warn | 1
diff --git a/website/src/content/docs/docs/concepts/adapters/clickhouse.mdx b/website/src/content/docs/docs/concepts/adapters/clickhouse.mdx
index 5aa6becff3..78ead20622 100644
--- a/website/src/content/docs/docs/concepts/adapters/clickhouse.mdx
+++ b/website/src/content/docs/docs/concepts/adapters/clickhouse.mdx
@@ -174,3 +174,11 @@ ClickHouse returns a type's default value (`0`, the epoch, an empty string) from
- Snapshots always use an unsorted `MergeTree` table.
- Seeds load through the HTTP insert interface as CSV, so large seeds never hit the `max_query_size` limit.
- Retention and time travel are not available.
+
+## Recall store
+
+A ClickHouse connection can also hold [Recall](/docs/concepts/observability/recall)'s command
+history for a whole team, whichever adapter the project builds with. Set
+`[recall] connection` to it; Recall creates its tables on first use, keeps every output byte by
+default, and needs only `SELECT` and `INSERT` once the tables exist. See
+[Sharing history in ClickHouse](/docs/concepts/observability/recall#sharing-history-in-clickhouse).
diff --git a/website/src/content/docs/docs/concepts/adapters/sqlite.mdx b/website/src/content/docs/docs/concepts/adapters/sqlite.mdx
index a1930765ee..d9f871d197 100644
--- a/website/src/content/docs/docs/concepts/adapters/sqlite.mdx
+++ b/website/src/content/docs/docs/concepts/adapters/sqlite.mdx
@@ -117,3 +117,9 @@ as `datetime(ordered_at, 'start of day')`, have no DuckDB translation and fail l
| SQLite older than 3.44 | `A701` |
| Another process holds the write lock beyond `busy_timeout` | `A702` |
| Concurrent microbatch writes: one file has one writer | refused at plan time |
+
+## Recall store
+
+[Recall](/docs/concepts/observability/recall) keeps its local history in a SQLite file,
+`logs/recall.sqlite3`, for every project, whichever adapter it builds with. Point
+`[recall] connection` at a `sqlite` connection to put the file somewhere else.
diff --git a/website/src/content/docs/docs/concepts/observability.mdx b/website/src/content/docs/docs/concepts/observability.mdx
index 7819885ff9..c93bfa395a 100644
--- a/website/src/content/docs/docs/concepts/observability.mdx
+++ b/website/src/content/docs/docs/concepts/observability.mdx
@@ -10,14 +10,28 @@ whether work started, completed, failed, skipped, or retried.
|--------|---------|-----------|
| `LifecycleEvent` | Immutable invocation, run, resource-attempt, operation, statement, retry, and audit facts | Canonical lifecycle evidence once durably stored |
| `DiagnosticLog` | Structured framework diagnostics | Explanatory only |
-| `stdout.log` / `stderr.log` | Exact host-local process output | Troubleshooting transcript |
-| [Recall](/docs/concepts/observability/recall) | Local queryable history of invocations, statements, SQL, and output | Troubleshooting history; best effort |
+| [Recall](/docs/concepts/observability/recall) | Queryable history of invocations, statements, SQL, audit results, and output, locally or in ClickHouse | Troubleshooting history; best effort |
+| `stdout.log` / `stderr.log` | Exact host-local process output (compute log files) | Troubleshooting transcript; also stored by Recall |
| `CommandOutputRecord` | Bounded, ANSI-free remote stdout/stderr chunks | Potentially sensitive and lossy troubleshooting data |
| Final JSON output | End-of-command aggregate | Result projection, not event history |
Human-readable CLI output remains the normal operator interface. Structured records are a separate
contract for integrations and durable consumers.
+## Which to use
+
+| You want to | Use |
+|-------------|-----|
+| Find out what a past run did: what ran, its SQL, audit results, errors and output | [Recall](/docs/concepts/observability/recall) and [`sqb recall`](/docs/cli/recall) |
+| Send events or output to your own systems, such as a message queue or a monitoring tool | [Typed sinks](/docs/concepts/observability/sinks) |
+| Store lifecycle events and query run state from your own Python application | [Execution history](/docs/concepts/observability/execution-history) |
+| Read the result of the command you just ran | Its final `--json` output |
+
+Recall is on by default and covers investigation for people and agents. Sinks and execution history
+are for building on SQLBuild's events elsewhere. The `stdout.log`, `stderr.log` and
+`diagnostics.jsonl` compute log files under `logs//` are still written alongside Recall and
+hold the same bytes; prefer `sqb recall output`, which works the same with a shared store.
+
## Lifecycle envelope
New events use lifecycle schema version 2:
diff --git a/website/src/content/docs/docs/concepts/observability/execution-history.mdx b/website/src/content/docs/docs/concepts/observability/execution-history.mdx
index 6291fdd6c8..95dcb01931 100644
--- a/website/src/content/docs/docs/concepts/observability/execution-history.mdx
+++ b/website/src/content/docs/docs/concepts/observability/execution-history.mdx
@@ -8,6 +8,10 @@ published, rebuildable run projection. Diagnostic and compute logs are never acc
contract. The public types, filters, pages, records, errors, and helper functions are exported from
`sqlbuild.execution_history`.
+This is a Python API for applications that store and serve SQLBuild events themselves. To look into
+what past runs did, use [Recall](/docs/concepts/observability/recall), which records every command
+automatically and is queried with [`sqb recall`](/docs/cli/recall).
+
## Public contracts
`LifecycleEventLogStorage` provides `append_event`, `append_events`, `get_events`, `get_schema_version`,
diff --git a/website/src/content/docs/docs/concepts/observability/recall.mdx b/website/src/content/docs/docs/concepts/observability/recall.mdx
index 69a1328e64..d0ac97d210 100644
--- a/website/src/content/docs/docs/concepts/observability/recall.mdx
+++ b/website/src/content/docs/docs/concepts/observability/recall.mdx
@@ -7,6 +7,64 @@ Recall records each `sqb` invocation in a local SQLite store, or in a shared
[ClickHouse store](#sharing-history-in-clickhouse), so you can ask what happened after the fact: which command ran, how it ended, which statements it executed and how long they took, and
exactly what it printed. Recall is on by default and needs no setup.
+## An investigation, start to finish
+
+The waffle shop example's `daily_revenue` model has an audit, `revenue_is_non_negative`, at warn
+severity. One night the payments feed starts recording a refund as a negative payment. The unit tests
+still pass, because they run on mocked data. The build finishes with a warning that nobody reads.
+
+The next morning, ask when the audit started failing:
+
+```bash
+sqb recall query --format table \
+ "SELECT invocation_started_at AS run, outcome, violation_count FROM audit_results
+ WHERE audit_name = 'revenue_is_non_negative' ORDER BY run DESC LIMIT 3"
+```
+
+```
+run | outcome | violation_count
+--------------------------- | ------- | ---------------
+2026-10-20T00:01:44.784683Z | warn | 1
+2026-10-20T00:00:34.652468Z | pass | 0
+2026-10-19T23:59:23.827831Z | pass | 0
+
+3 rows
+```
+
+See what that run printed, exactly as the scheduler saw it:
+
+```bash
+sqb recall output last --tail 4
+```
+
+```
+ daily_revenue
+ audit revenue_is_non_negative returned 1 row
+
+✓ Completed with warnings PASS=89 WARN=1 FAIL=0 INSUFFICIENT=0 SKIP=0 TOTAL=90 (3.08s)
+```
+
+Then check what is true in the warehouse now, with the same command:
+
+```bash
+sqb recall query --backend warehouse --format table \
+ "SELECT r.revenue_date, r.total_revenue_cents, p.payment_id, p.amount_cents
+ FROM prod.daily_revenue r JOIN prod.stg_payments p
+ ON CAST(p.paid_at AS DATE) = r.revenue_date
+ WHERE p.amount_cents < 0"
+```
+
+```
+revenue_date | total_revenue_cents | payment_id | amount_cents
+------------ | ------------------- | ---------- | ------------
+2026-04-03 | -1250 | 9 | -2200
+
+1 row
+```
+
+The bad row is still in prod, and payment 9 explains it. A coding agent can follow the same steps:
+every answer is a SQL query over the [views](#querying-history), with the same output in `--json`.
+
## What is recorded
For every recorded invocation, Recall keeps:
diff --git a/website/src/content/docs/docs/feature-comparison.mdx b/website/src/content/docs/docs/feature-comparison.mdx
index 01ad2944c0..1ae7e04db1 100644
--- a/website/src/content/docs/docs/feature-comparison.mdx
+++ b/website/src/content/docs/docs/feature-comparison.mdx
@@ -109,11 +109,11 @@ SQLBuild, dbt, and SQLMesh are all SQL pipeline frameworks. They share common gr
| Feature | SQLBuild | dbt | SQLMesh |
|---------|----------|-----|---------|
| Reference syntax | `__ref()` - parses as valid SQL | `{{ ref() }}` - Jinja template | `model_name` with dependency tracking |
-| Adapters | DuckDB, MotherDuck, Snowflake, BigQuery, Databricks, PostgreSQL, SQL Server | 30+ (community adapters) | DuckDB, Snowflake, BigQuery, Databricks, Spark, Redshift, Postgres, Trino, MySQL |
+| Adapters | DuckDB, MotherDuck, Snowflake, BigQuery, Databricks, PostgreSQL, SQL Server, ClickHouse, SQLite | 30+ (community adapters) | DuckDB, Snowflake, BigQuery, Databricks, Spark, Redshift, Postgres, Trino, MySQL |
| State requirements | Stateless by default | manifest.json + target/ | Requires state store (local database or PostgreSQL for production) |
| Playground | `sqb playground` | Clone example repo | Example project |
| AI agent skills | General guidance with `sqb skills`; Rules guidance with `sqb rules skills` | No | No |
-| Execution observability | Immutable schema-versioned lifecycle facts, invocation-local ordering, orchestration context, and typed project sinks | Events/artifacts plus external observability | Plans, state, and external observability |
+| Execution observability | Recall: every command's statements, SQL, audit results and output, queryable with SQL locally or in ClickHouse; immutable schema-versioned lifecycle facts and typed project sinks | Events/artifacts plus external observability | Plans, state, and external observability |
## Where each tool fits
@@ -125,4 +125,4 @@ SQLBuild, dbt, and SQLMesh are all SQL pipeline frameworks. They share common gr
## Not yet in SQLBuild
-- **Broader adapter support** - ClickHouse, Redshift, Trino, Spark, Athena
+- **Broader adapter support** - Redshift, Trino, Spark, Athena
diff --git a/website/src/content/docs/docs/index.mdx b/website/src/content/docs/docs/index.mdx
index 557da16103..8e68dbb6c4 100644
--- a/website/src/content/docs/docs/index.mdx
+++ b/website/src/content/docs/docs/index.mdx
@@ -42,6 +42,12 @@ updated, a [migration](/docs/concepts/models/migrations) moves its table instead
history, and the old name keeps working as a view. Columns can be renamed the same way. The
[janitor](/docs/cli/janitor) archives tables the project no longer builds before it deletes them.
+## Investigate
+
+[Recall](/docs/concepts/observability/recall) records every command: what ran, the SQL it executed,
+audit results and everything it printed. Agents and people query that history with SQL through
+[`sqb recall`](/docs/cli/recall), then check live state in the warehouse with the same command.
+
## How it fits together
| Step | Command | What it does |
@@ -51,8 +57,10 @@ history, and the old name keeps working as a view. Columns can be renamed the sa
| Plan | [`sqb plan`](/docs/cli/plan) | Shows what will run, why, and how much it will rebuild |
| Build | [`sqb build`](/docs/cli/build) | Runs tests, builds models in dependency order, and blocks bad data with audits |
| Compare | [`sqb diff`](/docs/cli/diff) | Compares schemas and rows between two targets |
+| Investigate | [`sqb recall`](/docs/cli/recall) | Queries the history of past runs, then live warehouse state |
-SQLBuild works with DuckDB, MotherDuck, Snowflake, BigQuery, Databricks, PostgreSQL and SQL Server.
+SQLBuild works with DuckDB, MotherDuck, Snowflake, BigQuery, Databricks, PostgreSQL, SQL Server,
+ClickHouse and SQLite.
DuckDB runs locally, so you can try everything without warehouse credentials. See
[Adapters](/docs/concepts/adapters) for connection setup.
diff --git a/website/src/content/docs/docs/roadmap.mdx b/website/src/content/docs/docs/roadmap.mdx
index 691c69a353..915215520d 100644
--- a/website/src/content/docs/docs/roadmap.mdx
+++ b/website/src/content/docs/docs/roadmap.mdx
@@ -18,8 +18,8 @@ A self-hosted web app for your SQLBuild project:
## Later
-- Moving BigQuery, Databricks and SQL Server from beta to supported.
-- More warehouses, such as ClickHouse, Redshift and Trino.
+- Moving BigQuery, ClickHouse, Databricks, SQLite and SQL Server from beta to supported.
+- More warehouses, such as Redshift and Trino.
## Not planned