HDDS-16020. Configure upgrade acceptance tests to run on master - #11039
Open
errose28 wants to merge 6 commits into
Open
HDDS-16020. Configure upgrade acceptance tests to run on master#11039errose28 wants to merge 6 commits into
errose28 wants to merge 6 commits into
Conversation
errose28
added
the
zdu
dombizita
approved these changes
Aug 26, 2026
dombizita
left a comment
dombizita
left a comment
Contributor
There was a problem hiding this comment.
Thanks @errose28, the restructure makes sense to me! Don't forget that hadoop-ozone/dist/src/main/compose/upgrade/test.sh has both suites running, not sure what you'd like to comment out.
Contributor
Author
|
Thanks for the review. I'll comment out the rolling one when this PR comes out of draft. For now I wanted to verify that both still ran after these changes. |
* HDDS-14496-zdu: (201 commits) HDDS-16025. Allow old clients to start finalization on a new server. (apache#11074) HDDS-16190. Do not use unified version framework in OM until ZDU is finalized. (apache#11025) HDDS-16154. Datanode should assign DatanodeDetails#currentVersion based on its apparent version (apache#11006) HDDS-16153. DatanodeDetails should use HDDSVersion instead of int (apache#10994) HDDS-15549. Recon's INITIAL_VERSION should not have an upgrade action (apache#10935) HDDS-15422. Stream read seek should not close stream (apache#10415) HDDS-16089. Improve MPU table evaluation efficiency in KeyLifeCycleService (apache#10949) HDDS-16169. Add old updateContainerState to handle upgrade path. (apache#11012) HDDS-16158. Fix sendDeleteKeysRequestAndClearList Authorization and FSO Trash Handling (apache#10996) HDDS-15911. Recon AI Assistant: add conversation memory for follow-up questions. (apache#10808) HDDS-15356. Make multi-buffer chunk checksum allocation-free (apache#10350) HDDS-15895. Make ServerNotLeaderException suggested leader flow IPv6-safe (apache#10807) HDDS-16118. Release snapshot DB handles before KeyDeletingService submits OM requests (apache#10987) HDDS-16125. Copy bucket snapshot in quota repair response to fix cache aliasing (apache#11004) HDDS-8082. Check for unnecessary newlines in config defaults (apache#10840) HDDS-15826. Recon: add cycle guard to NSSummary /du tree walks (apache#10723) HDDS-11620. Log SCM finalization completion (apache#10963) HDDS-16117. Failed FSO multipart complete leaks bucket namespace quota (apache#10984) HDDS-15961. Resolve linked bucket source properties consistently (apache#10859) HDDS-16072. Implement GetObjectAttributes: ETag, ObjectSize, StorageClass, and ObjectParts count (apache#10930). ...
Contributor
Author
|
The feature branch has all the changes needed to run these tests now. I forked a separate branch off this change to run just the rolling test to verify it works. This branch is now running just the nonrolling version as it will be committed to master until a ZDU release goes out. |
errose28
marked this pull request as ready for review
August 27, 2026 15:40
Contributor
Author
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Acceptance tests on the ZDU branch are currently doing a rolling restart within the same version, since Ozone must be upgraded to the first ZDU compatible version before starting a ZDU. On master, we will need to switch back to doing a non-rolling upgrade from the last release to latest master.
This change splits the old and new finalization CLIs into two different sets of robot files stored under
non-rollingandrollingupgrade subdirectories. This leaves the new CLIs reserved for the rolling upgrade flow and old CLIs for the non-rolling flow. This avoids adding a new split using the new CLIs for the non-rolling flow. Since new clients are backwards compatible with the old CLIs, we can continue to run non-rolling upgrade to the latest version using the old CLIs.This implementation requires HDDS-16025. Even though the clients are not old, they are using the old CLIs which are currently broken with the latest server. This also adds coverage for backwards compatibility with the old CLIs, which we currently don't have.
Other Changes
When testing rolling upgrade, only the first upgrade restart is done rolling. The subsequent downgrade and upgrade are done non-rolling to save CI time. The second downgrade/upgrade were covering the same combination of versions as the first upgrade anyways.
The HA-Proxy container we previously added to the upgrade suite was not ignored when there was an acceptance test failure and thread dumps were collected. This caused a chain reaction which failed CI artifact (log bundle) upload for the entire upgrade suite. This container is now ignored.
What is the link to the Apache JIRA
HDDS-16020
How was this patch tested?
This change is currently running both the rolling and non-rolling upgrade flows to make sure they both work. Once both are green on my fork we can comment out the rolling upgrade (currently just a restart) until the first release with ZDU support goes out.