diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/CONTENTS.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/CONTENTS.md index d34e58b97b..8f0d15cb00 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/CONTENTS.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/CONTENTS.md @@ -23,6 +23,9 @@ Bundled copies of every page on the SQLBuild documentation site, matching the in - [Project Configuration](concepts/project-configuration.md) (`concepts/project-configuration`) - Configure your SQLBuild project with sqlbuild_project.toml and sqlbuild_local.toml. - [Resource Identities](concepts/resource-identities.md) (`concepts/resource-identities`) - Canonical names for SQLBuild resources, selectors, state, and integrations. - [Semantic Compilation](concepts/semantic-compilation.md) (`concepts/semantic-compilation`) - Offline reference, semantic and type checks that sqb compile runs before any warehouse work. +- [Refactoring](concepts/refactoring.md) (`concepts/refactoring`) - Rename and move models and columns, keep their warehouse history, and keep old names working. +- [Model migrations](concepts/models/migrations.md) (`concepts/models/migrations`) - Keep a renamed model's history, and keep its old name working for a while. +- [Column migrations](concepts/models/column-migrations.md) (`concepts/models/column-migrations`) - Keep a column's history when you rename it in an incremental or snapshot model. - [Overview](concepts/adapters.md) (`concepts/adapters`) - Supported database engines and their connection configuration. - [Snowflake](concepts/adapters/snowflake.md) (`concepts/adapters/snowflake`) - Snowflake adapter configuration for SQLBuild. - [DuckDB](concepts/adapters/duckdb.md) (`concepts/adapters/duckdb`) - DuckDB adapter configuration for SQLBuild. @@ -38,8 +41,6 @@ Bundled copies of every page on the SQLBuild documentation site, matching the in - [Schemas](concepts/models/schemas.md) (`concepts/models/schemas`) - Declare model columns inline or reuse canonical inherited schemas. - [Type Enforcement](concepts/models/type-enforcement.md) (`concepts/models/type-enforcement`) - Understand declared model types, static checks, and runtime casting. - [Contracts](concepts/models/contracts.md) (`concepts/models/contracts`) - Validate required or exact model output schemas. -- [Model migrations](concepts/models/migrations.md) (`concepts/models/migrations`) - Keep a renamed model's history, and keep its old name working for a while. -- [Column migrations](concepts/models/column-migrations.md) (`concepts/models/column-migrations`) - Keep a column's history when you rename it in an incremental or snapshot model. - [Hooks](concepts/models/hooks.md) (`concepts/models/hooks`) - Run SQL or Python lifecycle hooks around model materialization. - [SQL Hooks](concepts/models/hooks/sql.md) (`concepts/models/hooks/sql`) - Define, parameterize, compile, and invoke reusable or inline SQL lifecycle hooks. - [Python Hooks](concepts/models/hooks/python.md) (`concepts/models/hooks/python`) - Define Python lifecycle hooks with runtime context, providers, SQL access, and skips. diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/refactoring.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/refactoring.md new file mode 100644 index 0000000000..4eadfaf1cb --- /dev/null +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/concepts/refactoring.md @@ -0,0 +1,118 @@ + + +# Refactoring + +> Rename and move models and columns, keep their warehouse history, and keep old names working. + +Online: https://sqlbuild.com/docs/concepts/refactoring/ + +## Contents + +- Rename a model +- Rename a column +- Old names keep working +- Check the impact first +- When a rename is refused + +SQLBuild lets you rename a model, move it to another folder, or rename one of its columns as a single +verified change. The compiler finds every reference, the edited project is compiled before anything is +written, and the next build keeps the existing table's history instead of rebuilding it. + +## Rename a model + +```bash +sqb rename hourly_order_activity order_activity_by_hour +``` + +```text +Rename model hourly_order_activity -> order_activity_by_hour +├── models/marts/daily_activity_rollup.sql +│ ├── 10:5 header hourly_order_activity -> order_activity_by_hour +│ └── 34:13 reference __ref("hourly_order_activity") -> __ref("order_activity_by_hour") +├── models/marts/hourly_activity_with_daily_context.sql +│ ├── 10:5 header hourly_order_activity -> order_activity_by_hour +│ └── 36:15 reference __ref("hourly_order_activity") -> __ref("order_activity_by_hour") +└── models/marts/order_activity_by_hour.sql (moved from models/marts/hourly_order_activity.sql) + └── 1:8 migration + migrate_from hourly_order_activity +Compiled: ok, 3 files changed +``` + +The plan then shows a migration instead of a first run: + +```bash +sqb plan +``` + +```text +Migrations (1) +└── order_activity_by_hour migrate dev.hourly_order_activity -> dev.order_activity_by_hour + ├── compatibility compatible + ├── transfer physical copy, promote by transactional rename + └── old name dev.hourly_order_activity + ├── view until 2026-11-06 + └── old table archived +``` + +On the next `sqb build`, the incremental model's existing data moves to the new name and the model +carries on from where it was. Nothing is backfilled. + +`sqb mv` does the same for a move to another folder, optionally with a new name: + +```bash +sqb mv fact_orders models/reporting/ +sqb mv fact_orders models/reporting/order_facts.sql +``` + +A move also takes the model's private macros, enums, constants and other scoped declarations with it, +following the same placement rules as `sqb compile`. See [Declarations and scopes](declaration-scopes.md). + +## Rename a column + +```bash +sqb rename stg_customers.email email_address --dry-run +``` + +```text +Rename column stg_customers.email -> email_address +├── models/marts/dim_customers.sql +│ ├── 16:5 column email -> email_address +│ ├── 16:10 column + AS email +│ └── 23:54 column email -> email_address +└── models/staging/stg_customers.sql + ├── 7:5 header email -> email_address + └── 15:8 column + AS email_address +Dry run: compiles, 2 files would change; nothing written +``` + +Edits follow the columns each query actually binds, so a same-named column of another table is left +alone. Downstream models keep their own output names unless you pass `--cascade`. An incremental or +snapshot model renames the column in place on the next build. See +[Column migrations](models/column-migrations.md). + +## Old names keep working + +After a model moves to a new name, the old name becomes a view over the new relation for 30 days by +default, with the old relation's grants copied to it. Queries, dashboards and other tools that read +the old name keep working while you update them. After that, [`sqb janitor`](../cli/janitor.md) drops +the view. See [Model migrations](models/migrations.md#old-names) for the settings. + +## Check the impact first + +- [`sqb lineage`](../cli/lineage.md) shows which models and columns depend on what you are changing. +- [`sqb scope`](../cli/scope.md) shows which declarations a model uses and what a move would break. +- `--dry-run` on `sqb rename` and `sqb mv` prints every edit and checks the edited project compiles, + without writing files. + +## When a rename is refused + +If any location cannot be rewritten safely, the command lists it and writes nothing: + +```text +Rename column stg_payments.amount_cents -> amount_minor +Cannot rename column:stg_payments.amount_cents automatically +└── models/marts/daily_revenue.sql:21:3 column amount_cents is referenced in SQL a macro generates; update the macro call by hand +Edit the listed locations, then run the command again. +Refused; no files changed +``` + +The full list of refusals and every flag is in the [`sqb rename` and `sqb mv` reference](../cli/rename.md). diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/index.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/index.md index 525ca6e3bf..16bd00797f 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/index.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/index.md @@ -33,9 +33,10 @@ real model graph against test data, then replay it locally on DuckDB in CI. [`sqb plan`](concepts/planning.md) shows why each model will run and how much it will rebuild. [`sqb diff`](concepts/diff.md) compares real data between targets before a change ships. When you -rename a model, a [migration](concepts/models/migrations.md) moves its table instead of rebuilding -its history, and the [janitor](cli/janitor.md) archives tables the project no longer builds before -it deletes them. +rename or move a model with [`sqb rename` and `sqb mv`](concepts/refactoring.md), every reference is +updated, a [migration](concepts/models/migrations.md) moves its table instead of rebuilding its +history, and the old name keeps working as a view. Columns can be renamed the same way. The +[janitor](cli/janitor.md) archives tables the project no longer builds before it deletes them. ## How it fits together diff --git a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/quickstart.md b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/quickstart.md index 827a81ca1a..77c7a7a981 100644 --- a/src/sqlbuild/.agents/skills/sqlbuild/references/docs/quickstart.md +++ b/src/sqlbuild/.agents/skills/sqlbuild/references/docs/quickstart.md @@ -215,6 +215,10 @@ sqb audit # Inspect lineage sqb lineage fact_orders --direction both +# Rename a model or a column, updating every reference (see Refactoring) +sqb rename fact_orders order_facts +sqb rename stg_customers.email email_address + # Compile and check contracts sqb compile ``` diff --git a/website/src/content/docs/docs/concepts/refactoring.mdx b/website/src/content/docs/docs/concepts/refactoring.mdx new file mode 100644 index 0000000000..65a9f9f448 --- /dev/null +++ b/website/src/content/docs/docs/concepts/refactoring.mdx @@ -0,0 +1,107 @@ +--- +title: "Refactoring" +description: "Rename and move models and columns, keep their warehouse history, and keep old names working." +--- + +SQLBuild lets you rename a model, move it to another folder, or rename one of its columns as a single +verified change. The compiler finds every reference, the edited project is compiled before anything is +written, and the next build keeps the existing table's history instead of rebuilding it. + +## Rename a model + +```bash +sqb rename hourly_order_activity order_activity_by_hour +``` + +```text +Rename model hourly_order_activity -> order_activity_by_hour +├── models/marts/daily_activity_rollup.sql +│ ├── 10:5 header hourly_order_activity -> order_activity_by_hour +│ └── 34:13 reference __ref("hourly_order_activity") -> __ref("order_activity_by_hour") +├── models/marts/hourly_activity_with_daily_context.sql +│ ├── 10:5 header hourly_order_activity -> order_activity_by_hour +│ └── 36:15 reference __ref("hourly_order_activity") -> __ref("order_activity_by_hour") +└── models/marts/order_activity_by_hour.sql (moved from models/marts/hourly_order_activity.sql) + └── 1:8 migration + migrate_from hourly_order_activity +Compiled: ok, 3 files changed +``` + +The plan then shows a migration instead of a first run: + +```bash +sqb plan +``` + +```text +Migrations (1) +└── order_activity_by_hour migrate dev.hourly_order_activity -> dev.order_activity_by_hour + ├── compatibility compatible + ├── transfer physical copy, promote by transactional rename + └── old name dev.hourly_order_activity + ├── view until 2026-11-06 + └── old table archived +``` + +On the next `sqb build`, the incremental model's existing data moves to the new name and the model +carries on from where it was. Nothing is backfilled. + +`sqb mv` does the same for a move to another folder, optionally with a new name: + +```bash +sqb mv fact_orders models/reporting/ +sqb mv fact_orders models/reporting/order_facts.sql +``` + +A move also takes the model's private macros, enums, constants and other scoped declarations with it, +following the same placement rules as `sqb compile`. See [Declarations and scopes](/docs/concepts/declaration-scopes). + +## Rename a column + +```bash +sqb rename stg_customers.email email_address --dry-run +``` + +```text +Rename column stg_customers.email -> email_address +├── models/marts/dim_customers.sql +│ ├── 16:5 column email -> email_address +│ ├── 16:10 column + AS email +│ └── 23:54 column email -> email_address +└── models/staging/stg_customers.sql + ├── 7:5 header email -> email_address + └── 15:8 column + AS email_address +Dry run: compiles, 2 files would change; nothing written +``` + +Edits follow the columns each query actually binds, so a same-named column of another table is left +alone. Downstream models keep their own output names unless you pass `--cascade`. An incremental or +snapshot model renames the column in place on the next build. See +[Column migrations](/docs/concepts/models/column-migrations). + +## Old names keep working + +After a model moves to a new name, the old name becomes a view over the new relation for 30 days by +default, with the old relation's grants copied to it. Queries, dashboards and other tools that read +the old name keep working while you update them. After that, [`sqb janitor`](/docs/cli/janitor) drops +the view. See [Model migrations](/docs/concepts/models/migrations#old-names) for the settings. + +## Check the impact first + +- [`sqb lineage`](/docs/cli/lineage) shows which models and columns depend on what you are changing. +- [`sqb scope`](/docs/cli/scope) shows which declarations a model uses and what a move would break. +- `--dry-run` on `sqb rename` and `sqb mv` prints every edit and checks the edited project compiles, + without writing files. + +## When a rename is refused + +If any location cannot be rewritten safely, the command lists it and writes nothing: + +```text +Rename column stg_payments.amount_cents -> amount_minor +Cannot rename column:stg_payments.amount_cents automatically +└── models/marts/daily_revenue.sql:21:3 column amount_cents is referenced in SQL a macro generates; update the macro call by hand +Edit the listed locations, then run the command again. +Refused; no files changed +``` + +The full list of refusals and every flag is in the [`sqb rename` and `sqb mv` reference](/docs/cli/rename). diff --git a/website/src/content/docs/docs/index.mdx b/website/src/content/docs/docs/index.mdx index b487a8e5f0..8af9dca007 100644 --- a/website/src/content/docs/docs/index.mdx +++ b/website/src/content/docs/docs/index.mdx @@ -33,9 +33,10 @@ real model graph against test data, then replay it locally on DuckDB in CI. [`sqb plan`](/docs/concepts/planning) shows why each model will run and how much it will rebuild. [`sqb diff`](/docs/concepts/diff) compares real data between targets before a change ships. When you -rename a model, a [migration](/docs/concepts/models/migrations) moves its table instead of rebuilding -its history, and the [janitor](/docs/cli/janitor) archives tables the project no longer builds before -it deletes them. +rename or move a model with [`sqb rename` and `sqb mv`](/docs/concepts/refactoring), every reference is +updated, a [migration](/docs/concepts/models/migrations) moves its table instead of rebuilding its +history, and the old name keeps working as a view. Columns can be renamed the same way. The +[janitor](/docs/cli/janitor) archives tables the project no longer builds before it deletes them. ## How it fits together diff --git a/website/src/content/docs/docs/quickstart.mdx b/website/src/content/docs/docs/quickstart.mdx index 80cb13f15d..a1da26990a 100644 --- a/website/src/content/docs/docs/quickstart.mdx +++ b/website/src/content/docs/docs/quickstart.mdx @@ -201,6 +201,10 @@ sqb audit # Inspect lineage sqb lineage fact_orders --direction both +# Rename a model or a column, updating every reference (see Refactoring) +sqb rename fact_orders order_facts +sqb rename stg_customers.email email_address + # Compile and check contracts sqb compile ``` diff --git a/website/src/sidebar.json b/website/src/sidebar.json index 410841f956..880f2e64cc 100644 --- a/website/src/sidebar.json +++ b/website/src/sidebar.json @@ -23,6 +23,18 @@ "docs/concepts/project-configuration", "docs/concepts/resource-identities", "docs/concepts/semantic-compilation", + { + "label": "Refactoring", + "collapsed": false, + "items": [ + { + "label": "Overview", + "slug": "docs/concepts/refactoring" + }, + "docs/concepts/models/migrations", + "docs/concepts/models/column-migrations" + ] + }, { "label": "Adapters", "collapsed": true, @@ -48,8 +60,6 @@ "docs/concepts/models/schemas", "docs/concepts/models/type-enforcement", "docs/concepts/models/contracts", - "docs/concepts/models/migrations", - "docs/concepts/models/column-migrations", { "label": "Hooks", "collapsed": true,