Repository navigation
feat(migrations): keep old model names working with expiring compatibility views - #871
Merged
Merged
Conversation
…ility views Model migrations now archive the old relation after the destination builds and create a view at the old name that reads the new relation, with column renames aliased back to their old names. Tables and views accept migrate_from and use the same migration path. State is recorded as append-only, deterministic-ID facts in a new _sqlbuild_old_name_views table; a required fact gates the new behaviour so moves recorded before the upgrade keep today's behaviour. A model that would replace a compatibility view is a planning error (M114). Configured with [migrations] old_name_views (default 30d) and the old_name_view model header; false keeps today's behaviour. Refs CHI-391
…eads The janitor lists compatibility views at old model names, drops them once they expire or when named with --drop-old-name-view, and records each drop as an append-only fact. General cleanup no longer archives a live compatibility view. P008 now also rejects literal Python SQL that reads a declared old name at compile time, and a discovered old name at plan time. Refs CHI-391
A failure while archiving the old relation or creating its compatibility view now fails the model with M117 and its resume hint instead of the generic worker error. View publication is isolated in one step so later privilege handling has a single place to attach. Refs CHI-391
Rewrites the migration contract that the old relation is never modified, documents old-name views, their retention and janitor handling, the [migrations] config and old_name_view header, and the upgrade note that janitor and builds should run the same version. Refs CHI-391
…ndow Refs CHI-391
…tion transaction Compatibility views bound to a model's relation are no longer dropped before the build and restored afterwards. The swap, build-aside rename, and column drop or type change of the destination now run in one transaction with the views' rebinding, so a crash at any point leaves each view present with its old or new definition. Views are replaced in place when their columns allow it, so views stacked on them outside SQLBuild keep working. Nested adapter transactions now join the outer transaction. Refs CHI-391
…n every adapter The compatibility view at a renamed model's old name gets the privileges that were granted on the old relation, read from its archive with each warehouse's own mechanism: Snowflake SHOW GRANTS (skipping OWNERSHIP and shares, keeping grant option), PostgreSQL table ACLs, BigQuery table IAM bindings, Unity Catalog grants, and SQL Server object permissions. Nothing new is granted. The statements are recorded on view_created as the additive grants_copied column and reapplied whenever the view is re-created. Old-name views now run on BigQuery, Databricks and SQL Server too, so the supports_old_name_views capability and its plan note are removed. BigQuery cannot rename views, so an old view model is re-created at the archive name. Refs CHI-391
sqb plan nests the old name under each migration with one row each for the view expiry, column aliases, copied grants and the archive. The janitor lists compatibility views as a tree with their expiry or drop reason, and plan JSON gains old_name.grants_copied. Refs CHI-391
…adapter Refs CHI-391
…sh windows Postgres crash tests now close the session at the fault, as a killed process would, so nothing after the fault can clean up; every crash point leaves the compatibility view readable. Refs CHI-391
…table has Readers inspect the stored _sqlbuild_old_name_views schema and select only the known columns it has, reading columns added later as NULL, as the additive state-schema rule requires. Refs CHI-391
… steps A move whose archive or view is still pending no longer resumes when the old name is now a project destination, is a compatibility view of another move, or the model has turned old-name views off; nothing is archived or created and the plan shows the reason under 'left for janitor'. Refs CHI-391
…es, rebuilds, and archives - Adapters read grants as structured facts and render them for the view's exposed columns. SQL Server now copies column-level GRANT and DENY, and PostgreSQL copies column privileges; a column the view does not expose is skipped. - Redefining a view keeps its current privileges instead of replaying the grants recorded at creation: CREATE OR REPLACE on PostgreSQL, COPY GRANTS on Snowflake, ALTER VIEW on SQL Server and Databricks. BigQuery, and a PostgreSQL view that must be dropped, re-read the view's grants just before and reapply them. - Builds redefine only views that exist; a recorded view that is absent is left for janitor's missing reconcile. - BigQuery copies a view's IAM bindings to the archive when archiving a view model. - BigQuery warns with M118 which principals kept access to the old name and also need read access on the destination table; nothing is granted. Refs CHI-391
…ity view's name When a recorded view's name is a project model's destination and that model has a fingerprint there newer than the view's creation, the relation is the project's: janitor runs no DDL, records the view as dropped with reason missing, and lists 'name now built by model:x'. Otherwise the view is still SQLBuild's, so expiry and --drop-old-name-view proceed, keeping the M114 remedy. Drops re-check this evidence just before running. Refs CHI-391
…esuming A resumed archive or view no longer requires the origin's own fingerprint, which the destination build of a same-model alias move supersedes; the recorded move and its required fact prove SQLBuild owned the origin. Resume keeps the name-reuse, compatibility-view, and disabled checks. Refs CHI-391
… view grants exactly - Janitor drops a recorded compatibility view only when the relation at the old name is a view whose stored definition matches the SQL SQLBuild renders for the move, or the SQL recorded on view_created (new additive view_sql column). Any other relation is left alone, listed as 'name now used by another relation', and the view is recorded as missing. Drops re-check just before running; fingerprints are no longer consulted. PostgreSQL compares through a rolled-back temporary view, since it rewrites stored definitions. - Every create, replace, or re-create of a compatibility view reconciles its privileges to the ones it had just before (or the old relation's, for a new view): missing grants are granted and grants added by default privileges or schema future grants are revoked. Owner privileges are never read or changed. - On adapters whose views bind by name at query time, a build skips redefining a view whose SQL is unchanged. Refs CHI-391
… project model now uses A recorded compatibility view whose old name is a project model's destination (for example after an interrupted early drop) is no longer refreshed, rebound, or released by builds of the destination. Before, when the project's model built at that name before the destination, the destination's refresh overwrote the project's view with the compatibility SQL; concurrent builds made that order likely. Refs CHI-391
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Renaming or moving a model breaks every consumer outside the project that still queries the old name. Teams avoided renames or hand-wrote compatibility views.
Changes
[migrations] old_name_views, modelold_name_view,falsekeeps today's behaviour).migrate_fromand automatic rename discovery now cover table and view models (outcomes migrate, done, origin missing)._sqlbuild_old_name_viewswith deterministic IDs; every failure point resumes without repeating work. Postgres rebinds SQLBuild's views inside promotion transactions.--drop-old-name-view; a model claiming a live view's name fails planning (M114); hard-coded old names are P008 errors.sqb planshows a nestedold nameblock; plan JSON gainsold_name.Verification
make test,make check-ciand the hygiene scan pass; one independent review.