Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
170a3ef
feat(migrations): keep old model names working with expiring compatib…
kvlonge Sep 28, 2026
768845d
feat(janitor): drop expired compatibility views and reject old-name r…
kvlonge Sep 28, 2026
df65817
fix(migrations): report old-name step failures as M117
kvlonge Sep 28, 2026
3eff912
feat(docs): document compatibility views at renamed models' old names
kvlonge Sep 28, 2026
374fd8b
test(migrations): cover old-name views end to end and every resume wi…
kvlonge Sep 28, 2026
dce683e
fix(migrations): rebind Postgres compatibility views inside the promo…
kvlonge Sep 28, 2026
08f2d5a
feat(migrations): copy old relations' grants to compatibility views o…
kvlonge Sep 28, 2026
5c573b4
feat(plan): show each old-name fact on its own row
kvlonge Sep 28, 2026
d7bb2a8
docs(migrations): document copied grants and old-name views on every …
kvlonge Sep 28, 2026
8bf22af
test(migrations): cover copied grants, nested plan rows, and real cra…
kvlonge Sep 28, 2026
6e49d4a
fix(migrations): read old-name view facts from the columns the state …
kvlonge Sep 28, 2026
a55ab66
fix(migrations): apply the old-name skip checks when resuming pending…
kvlonge Sep 28, 2026
f6744f5
fix(migrations): keep compatibility view privileges exact across copi…
kvlonge Sep 28, 2026
812eb94
fix(janitor): never drop a relation the project built at a compatibil…
kvlonge Sep 28, 2026
a81b73c
fix(migrations): trust the recorded move as ownership evidence when r…
kvlonge Sep 28, 2026
1a554ec
fix(migrations): decide view ownership from definitions and reconcile…
kvlonge Sep 28, 2026
2edacde
fix(migrations): never redefine a compatibility view whose old name a…
kvlonge Sep 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions fensu.toml
Original file line number Diff line number Diff line change
Expand Up @@ -278,3 +278,20 @@ contract = "src/sqlbuild/adapter/contract/classes/strict_adapter.py:StrictAdapte
forbidden_owners = ["src/sqlbuild/adapter/contract/classes/base_adapter.py:BaseAdapter"]
paths = ["src/sqlbuild/adapter/contract/classes/*", "src/sqlbuild/adapters/*/classes/*"]
reason = "tests/unit/src/sqlbuild/adapter/contract/classes/strict_adapter/test_strict_adapter.py requires every first-class adapter to define each StrictAdapter method itself instead of inheriting it from BaseAdapter."

[[targets.python.rule_exceptions]]
rule = "XSB051"
path = "src/sqlbuild/executor/migrations/_helpers/old_name_views.py"
symbols = [
"present_old_name_views",
"recreate_old_name_views",
"refresh_old_name_views",
"run_planned_old_name_steps",
]
reason = "Old-name steps re-read facts and physical state per move so a retry never repeats a completed step, and compatibility views are re-created in dependency order, so a chained view must read the columns of the view it selects from after that view is re-created. Each view's existence is checked right before it is redefined, and a model has only its own short chain of old names."

[[targets.python.rule_exceptions]]
rule = "XSB051"
path = "src/sqlbuild/executor/old_name_views/_helpers/janitor.py"
symbols = ["plan_old_name_views", "apply_old_name_view_drops"]
reason = "Janitor reads each compatibility view's stored definition right before deciding to drop it, so a relation that replaced the view is never dropped; a project has few live old names."
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Bundled copies of every page on the SQLBuild documentation site, matching the in
- [Schemas](concepts/models/schemas.md) (`concepts/models/schemas`) - Declare model columns inline or reuse canonical inherited schemas.
- [Type Enforcement](concepts/models/type-enforcement.md) (`concepts/models/type-enforcement`) - Understand declared model types, static checks, and runtime casting.
- [Contracts](concepts/models/contracts.md) (`concepts/models/contracts`) - Validate required or exact model output schemas.
- [Model migrations](concepts/models/migrations.md) (`concepts/models/migrations`) - Keep the history of an incremental or snapshot model when you rename it.
- [Model migrations](concepts/models/migrations.md) (`concepts/models/migrations`) - Keep a renamed model's history, and keep its old name working for a while.
- [Column migrations](concepts/models/column-migrations.md) (`concepts/models/column-migrations`) - Keep a column's history when you rename it in an incremental or snapshot model.
- [Hooks](concepts/models/hooks.md) (`concepts/models/hooks`) - Run SQL or Python lifecycle hooks around model materialization.
- [SQL Hooks](concepts/models/hooks/sql.md) (`concepts/models/hooks/sql`) - Define, parameterize, compile, and invoke reusable or inline SQL lifecycle hooks.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Online: https://sqlbuild.com/docs/cli/janitor/
- Usage
- Flags
- Lifecycle
- Compatibility views
- Relation age
- Audit record
- Restoring an archive
Expand All @@ -34,6 +35,7 @@ sqb --project-dir <path> janitor [flags]
| `--auto-approve` | Skip the confirmation prompt and run the planned actions immediately |
| `--retention-days` | Override the configured retention period (days) before a stale relation is archived |
| `--direct-state-history-versions` | Override how many state-history versions are kept per identity |
| `--drop-old-name-view <name>` | Drop the [compatibility view](../concepts/models/migrations.md#old-names) at a renamed model's old name before it expires. Repeat for several views. Fails with `C503` if no compatibility view is recorded at the name |

## Lifecycle

Expand All @@ -58,6 +60,20 @@ Long names are shortened with a short hash so they fit the adapter's identifier

Setting both `retention_days` and `archive_retention_days` to `0` archives and deletes stale relations in the same run.

## Compatibility views

When a model is renamed, SQLBuild keeps a [compatibility view](../concepts/models/migrations.md#old-names) at its old name. The janitor lists each one with the model it reads and its expiry:

```
Old name views (1)
└── prod.revenue -> model:daily_revenue
└── expires 2026-10-28 (live; drop early with --drop-old-name-view prod.revenue)
```

A compatibility view is never archived as a stale relation. Once it expires, or when it is named with `--drop-old-name-view`, the janitor drops it and records the drop in `_sqlbuild_old_name_views`. Before dropping, the janitor checks that the relation at the old name is still the view SQLBuild created there, by comparing its stored definition with the view's SQL. Any other relation at that name, such as a view a project model built after an interrupted early drop, is left alone: the janitor records the compatibility view as dropped and lists it as `record dropped (name now used by another relation, model:revenue)`, naming the model when the name is a project destination. A compatibility view that no longer exists is recorded as dropped too.

Run the same SQLBuild version for `sqb janitor` as for builds. An older janitor does not know compatibility views and archives them as stale relations.

## Relation age

`retention_days` is measured from the last time a relation was written, as reported by the warehouse. For a relation the project no longer builds, that is roughly when it stopped being built.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ After the rename is recorded, `migrate_from` has no further effect, and SQLBuild

A declaration is rejected when compiling if:

- the model is not incremental or snapshot;
- the model is not incremental or snapshot, unless it is a table or view model that declares its own `migrate_from`; there the column's `migrate_from` only names the old column in the [compatibility view](migrations.md#old-names) at the model's old name;
- `migrate_from` names the column itself, or a qualified name instead of one column;
- two columns declare the same `migrate_from`;
- a declared column migrates from another column that itself declares `migrate_from`. Chains and swaps can't be renamed in place safely; rename through a name no other column uses, or rebuild with `--full-refresh`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Online: https://sqlbuild.com/docs/concepts/models/configuration/
| `materialized` | `view`, `table`, `incremental`, `snapshot`, or a custom materialization name |
| `tags` | Tags used by selectors |
| `description` | Human-readable model description |
| `columns` | Model-local column declarations, inherited-column audit augmentation, or column renames with `migrate_from` (incremental and snapshot models; see [Column migrations](column-migrations.md)) |
| `columns` | Model-local column declarations, inherited-column audit augmentation, or column renames with `migrate_from` (incremental and snapshot models; see [Column migrations](column-migrations.md)). On a table or view model that declares its own `migrate_from`, a column `migrate_from` only names the old column in the [compatibility view](migrations.md#old-names) |
| `model_schema` | Reusable column schema name |
| `audits` | Model-level audit instances |
| `enums` | Model-local enum declarations; names must begin with `_` |
Expand Down Expand Up @@ -81,8 +81,9 @@ Table promotion mode is a project setting rather than a `MODEL()` field. Staged
| `full_refresh` | Optional model execution override: `false` always runs incrementally, `true` always full-refreshes, and omission follows the command |
| `on_schema_change` | `append_new_columns`, `sync_all_columns`, `ignore`, or `fail` |
| `replay_on_change` | `forward`, `full`, or `bounded-<duration>` |
| `migrate_from` | Old model name (or `schema.name`) whose data this model takes over; see [Model migrations](migrations.md). Also valid on snapshot models |
| `migrate_force` | `true` replaces a destination that already has its own build history during a migration |
| `migrate_from` | Old model name (or `schema.name`) whose data this model takes over; see [Model migrations](migrations.md). Also valid on snapshot, table, and view models |
| `migrate_force` | `true` replaces a destination that already has its own build history during a migration. Incremental and snapshot models only |
| `old_name_view` | How long a [compatibility view](migrations.md#old-names) keeps the old name working after a migration, such as `7d`, or `false` for none. Overrides `[migrations] old_name_views`; valid on every model |

See [Incremental](../incremental.md) for full semantics.

Expand Down
Loading
Loading