Repository navigation
feat: rename a project safely with sqt rename-project - #386
Merged
Merged
Conversation
…groups A project can set consumer_group_identity in sqltide_project.toml to use another value in place of name when deriving default Kafka consumer groups. Per-source consumer_group still wins, and leaving the key unset keeps today's groups and DDL unchanged. The value travels through the authored and effective configuration, the compile inputs and the compiled project to the managed source realization request. The shape baseline records the new trailing defaulted fields.
sqt rename-project NEW_NAME previews a project rename for the selected target and applies it with --apply: - keeps every Kafka consumer group: when a managed source uses the default group, consumer_group_identity is set to the old name, and the command re-derives every group before and after and refuses any change - copies the target's manifest history to the new identity under the target mutation lock, skipping manifests already copied, so destroy planning still finds relations recorded before the rename - moves project role assignments in the control store in one audited transaction - edits sqltide_project.toml last, splicing value spans so comments and formatting are kept; a failed write moves the role assignments back It refuses when the name is unchanged or invalid, a run holds the target mutation lock, the new name already has its own manifest history, or both names hold the same active role assignment.
…r group When a managed Kafka table's live kafka_group_name differs from the declared group, sqt plan (direct, virtual and mixed) prints a warning naming both groups and noting that offsets would not carry over if the table were recreated; --json lists it under consumer_group_warnings. The UI's source health shows the same warning. The check reuses the catalog the plan already read and changes no plan or build decision.
…ouse Builds a direct project with a managed Kafka source on a real ClickHouse target and checks that a rename keeps the Kafka table, consumer group and plan unchanged, copies manifest history visible to destroy planning, writes nothing on a dry run, refuses an unchanged name, completes after a lost config edit without duplicating manifests, refuses foreign manifest history and conflicting role grants, moves role grants, and reports consumer group drift in plan text, plan JSON and the UI state payload.
Add the sqt rename-project reference, a Project identity section with consumer_group_identity and Renaming a project to the project configuration guide, and the consumer group drift warning to the plan and Sources pages.
Each target has its own warehouse and usually its own UI host and control store, so a rename must be completable on the remaining targets after sqltide_project.toml already carries the new name. sqt rename-project NEW_NAME --from OLD_NAME leaves the config unchanged, checks that the target's Kafka consumer groups stay the same, copies the target's manifest history from OLD_NAME and moves the control store's role assignments from OLD_NAME, with the same lock, dry-run default, idempotence and conflict rules as the first run. It refuses when NEW_NAME is not the configured name, when the config is still named OLD_NAME, or when OLD_NAME equals the new name. The first run now names the --from command for the targets still to complete.
Document --from and the multi-target procedure, and note the dry run's control store edge cases.
- --from merges the old manifest history into a target that has already built under the new name instead of refusing it; the first run still refuses a new name with foreign manifest history - --from refuses when nothing on the target records OLD_NAME (no manifests, no role assignments), which usually means a mistyped name - a rename refuses while a managed Kafka table's live consumer group already differs from the declared one, instead of pinning consumer_group_identity to a name the live tables do not use - the control store URL is read from the invocation's environment - plan JSON, and the published plan artifact build shares, always carry consumer_group_warnings (an empty list when there is no drift), added without re-rendering the rest of the plan - the plan output and docs say role grants move for the whole control store, what that means for targets on the old config, that two projects with one name cannot share a store, and that connecting renames legacy metadata tables during a dry run
Completing a rename with --from OLD_NAME merged whatever history the target recorded under OLD_NAME. With two projects in one database, --from the other project's name copied its manifests, and destroy planning then treated its pipelines as retired pipelines of this project. Completing now requires consumer_group_identity, when set, to equal OLD_NAME, and otherwise the newest OLD_NAME manifest on the target to share a pipeline with the compiled project. The no-history refusal also names the legitimate case of a target never built under OLD_NAME, and the shared control store note is only printed when a control store is in use.
Deploying sqltide with
|
| Latest commit: |
fa27e1c
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://3ac7b758.sqltide.pages.dev |
| Branch Preview URL: | https://feat-project-rename.sqltide.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Renaming a project on a live target had no supported path. The default Kafka consumer group includes the project name, so a rename quietly put sources on a new group with no committed offsets (re-read or gap the next time a Kafka table is recreated), revoked UI role grants, and hid earlier manifests from
sqt destroy.Changes
consumer_group_identityinsqltide_project.tomlreplaces the project name in default consumer groups. Unset keeps today's DDL byte for byte.sqt rename-project NEW_NAME [--apply] [--from OLD_NAME]: dry run by default. Keeps every consumer group unchanged, copies manifest history under the target lock, moves control-store role grants, and edits the config last, preserving formatting.--fromcompletes the rename on each other target, including one already built under the new name, and refuses history from another project.sqt planand the UI warn when a Kafka table's live consumer group differs from the declared group. Plan JSON gainsconsumer_group_warnings.Verification
--from(with a build in between), dry run writes nothing, idempotent re-apply, conflict and wrong-project refusals, live drift refusal, drift warning in text, JSON and UI.Closes CHI-1210