Add seed migration steps to Rocky 10 migration documentation - #2553
Add seed migration steps to Rocky 10 migration documentation#2553owenjones wants to merge 1 commit into
Conversation
📝 WalkthroughUpdated Rocky 10 migration documentation with a complete seed-host migration procedure. The procedure covers service shutdown, volume backup and restoration, seed reprovisioning, secret-store redeployment, and seed-service deployment. The Azimuth operation-status check is now optional. WalkthroughThe Rocky 10 operations guide makes the Azimuth check conditional and replaces the seed migration TODO with procedures for service shutdown, volume backup and restoration, seed reprovisioning, and service deployment. ChangesRocky 10 migration guidance
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: 🔴 Critical · up to The Rocky 10 migration procedure can allow operators to destroy seed data before completing an off-seed backup, omit required Pulp storage, or lose the key material needed to restore the secret store. Merge should be blocked until the backup ordering, volume coverage, restore method, and key-file preservation requirements are explicit. 🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.) Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 492e5630-b335-4144-84e4-60fb94191115
📒 Files selected for processing (1)
doc/source/operations/rocky-10.rst
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.
📜 Review details
⏰ Context from checks skipped due to timeout. (3)
- GitHub Check: stackhpc/check
- GitHub Check: Ansible 2.18 lint with Python 3.12
- GitHub Check: Ansible 2.17 lint with Python 3.10
🔇 Additional comments (1)
doc/source/operations/rocky-10.rst (1)
20-20: LGTM!
| #. Stop seed services: | ||
|
|
||
| .. code-block:: console | ||
|
|
||
| kayobe seed service destroy --yes-i-really-really-mean-it |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🔴 Critical | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
#!/bin/bash
set -eu
printf '%s\n' '--- applicable repository conventions ---'
find /tmp/coderabbit-repo-knowledge/stackhpc-stackhpc-kayobe-config-578eaa86 -maxdepth 2 -type f -name '*.md' -print
printf '%s\n' '--- document context ---'
sed -n '450,525p' doc/source/operations/rocky-10.rst
printf '%s\n' '--- direct references ---'
rg -n -C 3 'seed service destroy|seed vm deprovision|backup|Pulp|secret|key.?file|volume' doc/source/operations/rocky-10.rstRepository: stackhpc/stackhpc-kayobe-config
Length of output: 4524
🌐 Web query:
Kayobe 2025.1 seed service destroy --yes-i-really-really-mean-it destroys containers images volumes
💡 Result:
Yes, the Kayobe command kayobe seed service destroy --yes-i-really-really-mean-it destroys containers, container images, and volumes that were deployed by Kayobe and Kolla [1][2][3]. According to official Kayobe documentation, this command is intended for cleaning up services on the seed host and is generally advised only when there is no important data on the system [1][4][5]. Key points regarding this command: * It permanently destroys all services and associated data deployed by Kayobe and Kolla [1][6]. * It does not automatically clean up volumes and images associated with custom containers [1][2]. To handle those, you must configure pre_destroy and post_destroy hooks to perform the cleanup manually [1][3][7]. * The --yes-i-really-really-mean-it flag is required as a confirmation that you understand the destructive nature of the command [6].
Citations:
- 1: https://docs.openstack.org/kayobe/2025.1/administration/seed.html
- 2: https://static.opendev.org/docs/kayobe/2025.1/administration/seed.html
- 3: https://docs.openstack.org/kayobe/2026.1/administration/seed.html
- 4: https://docs.openstack.org/kayobe/latest/administration/seed.html
- 5: https://docs.openstack.org/kayobe/2025.2/administration/seed.html
- 6: https://github.com/openstack/kayobe/blob/master/kayobe/cli/commands.py
- 7: https://docs.openstack.org/kayobe/latest/configuration/reference/seed-custom-containers.html
Back up the seed before running the destructive command.
kayobe seed service destroy --yes-i-really-really-mean-it destroys Kayobe and Kolla containers, images, and volumes. The backup step must precede this command, and the backup must be stored off the seed.
Source: MCP tools
There was a problem hiding this comment.
The bot is correct. It will destroy all your data before the backup.
There was a problem hiding this comment.
@jovial, thanks for confirming. The procedure must back up the required volumes and store the backup off the seed before kayobe seed service destroy --yes-i-really-really-mean-it. The destructive command can then run after the backup completes.
You are interacting with an AI system.
There was a problem hiding this comment.
Is there not a service stop? Backing up while containers are still running is probably not good.
There was a problem hiding this comment.
Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!
There was a problem hiding this comment.
I just stopped containers and the used virsh shutdown before copying the images from /var/lib/libvirt/images
There was a problem hiding this comment.
Perhaps provide a one liner to stop all containers. Remember to stop bifrost using systemctl stop.
| #. Take a backup of relevant container volumes on the seed: | ||
|
|
||
| #. Bifrost inventory | ||
| #. Secret store (Vault or OpenBao) | ||
| #. Pulp (root and data volumes) | ||
|
|
||
| #. Deprovision and reprovision the seed host: | ||
|
|
||
| .. code-block:: console | ||
|
|
||
| kayobe seed vm deprovision | ||
|
|
||
| .. code-block:: console | ||
|
|
||
| kayobe seed vm provision | ||
|
|
||
| #. Configure the seed: | ||
|
|
||
| .. code-block:: console | ||
|
|
||
| kayobe seed host configure | ||
|
|
||
| #. Restore the container volumes |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
List every Pulp mount in the backup and restore procedure.
The seed configuration mounts /opt/kayobe/containers/pulp, pulp_storage, pulp_pgsql, and pulp_containers. The checklist only says “Pulp (root and data volumes)” and the restore step gives no volume names or commands. An operator can therefore omit the Pulp database or registry storage and continue to deployment. Name every mount and document the supported backup and restore method. (raw.githubusercontent.com)
Source: MCP tools
| #. Restore the container volumes | ||
|
|
||
| #. Redeploy secret store: | ||
|
|
||
| .. code-block:: console | ||
|
|
||
| kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/secret-store/secret-store-deploy-seed.yml |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- repository conventions ---'
find /tmp/coderabbit-repo-knowledge/stackhpc-stackhpc-kayobe-config-578eaa86 -maxdepth 2 -type f -name '*.md' -print
printf '%s\n' '--- operations document ---'
sed -n '450,525p' doc/source/operations/rocky-10.rst
printf '%s\n' '--- secret-store deployment references ---'
rg -n -C 8 'seed-vault-keys|seed-openbao-keys|secret_store_keys|keys_base64|secret-store-deploy-seed' .Repository: stackhpc/stackhpc-kayobe-config
Length of output: 50387
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '--- scoped repository learning ---'
cat /tmp/coderabbit-repo-knowledge/stackhpc-stackhpc-kayobe-config-578eaa86/learnings/etc-kayobe-ansible-secret-store.md
printf '%s\n' '--- seed deployment playbook ---'
sed -n '1,110p' etc/kayobe/ansible/secret-store/secret-store-deploy-seed.yml
printf '%s\n' '--- secret-store configuration references ---'
rg -n -C 3 'seed-(vault|openbao)-keys\.json|vault_write_keys_file|openbao_write_keys_file|stackhpc_ca_secret_store' etc/kayobe doc/source/configuration/openbao.rstRepository: stackhpc/stackhpc-kayobe-config
Length of output: 42126
Preserve the secret-store key file.
If the procedure restores Vault/OpenBao volumes, it must also preserve the unchanged seed-vault-keys.json or seed-openbao-keys.json file in the Kayobe configuration. secret-store-deploy-seed.yml loads this file and uses secret_store_keys.keys_base64 to unseal the restored store. A missing or different file can prevent unsealing.
Source: MCP tools
|
@owenjones thanks for starting this. A few things worth mentioning from my observations upgrading the seed-vm elsewhere.
* Seed service deploy might fail as the host is now Rocky 10 which means Note: I am also submitting a patch to libvirt-vm role to support pulling overcloud images from Ark. This would allow for root LVM configuration. |
3 participants
No description provided.