fix: cloud control hotswap removes reserved aws: tags - #1940
Merged
Conversation
Contributor
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
dgandhi62
force-pushed
the
fix-tag-hotswap
branch
from
September 4, 2026 20:53
2d84a46 to
163db2f
Compare
A tags-only hotswap emitted a wholesale "replace /Tags", which declares the resource's complete desired tag set. On a CloudFormation-created resource that implies deleting the reserved aws:cloudformation tags, which the service rejects with "aws: prefixed tag key names are not allowed for external use". Address individual tags by their index in the resource's current Tags list instead, so reserved tags are never named by the patch. Indices come from a Cloud Control GetResource read; if that read fails we now raise HotswapTagReadFailed rather than falling back to the request that fails. Also re-enables the DynamicTag line in the cc-hotswap integ fixture, which was temporarily disabled to unblock CI.
A tags-only hotswap emitted a wholesale "replace /Tags", which declares the resource's complete desired tag set. On a CloudFormation-created resource that implies deleting the reserved aws:cloudformation tags, which the service rejects with "aws: prefixed tag key names are not allowed for external use". Address individual tags by their index in the resource's current Tags list instead, so reserved tags are never named by the patch. Indices come from a Cloud Control GetResource read; if that read fails we now raise HotswapTagReadFailed rather than falling back to the request that fails. Also re-enables the DynamicTag line in the cc-hotswap integ fixture, which was temporarily disabled to unblock CI.
rix0rrr
approved these changes
Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two
cli-integtests on the sharedcc-hotswapstack failed on every PR (tests 5 & 6), blocking the merge queue:hotswap deployment supports CloudControl-based resources with attribute resolutionhotswap deployment caches template and uses it for subsequent hotswapsBoth died on the
--hotswapdeploy with:Only the SQS Queue failed. The CloudWatch Dashboard and Events Rule in the same stack hotswapped successfully. Nothing in this repo seems to have caused this change - leading me to believe this is a service side behavioral change.
Why
The only thing the deploy changed on the Queue was a tag (the test fixture flips
DynamicTagbetween deploys). For a tag change, the hotswap code sent one "replace the whole Tags list" operation:json
Any tag already on the resource that isn't in this list gets dropped. The issue arose because CFN adds tags themselves on resources (aws:cloudformation:stack-name, stack-id, and logical-id). The underlying change now made it such that us sending the list of tags asked the service to remove the CFN ones as well, which it wouldn't allow because they are aws: prefixed. So it rejected the request.
Fix
We now address individual tags by their index in the resource's current
Tagslist, so reserved tags are never named by the patch and the service sees no change to them:[{ "op": "replace", "path": "/Tags/4", "value": { "Key": "DynamicTag", "Value": "new value" } }]The live ordering is not the template ordering, so the indices cannot be derived from the template. They come from reading current state with Cloud Control
GetResource. If it cannot be read, we raiseHotswapTagReadFailedKnown assumption
The approach assumes the live tag ordering is stable between the
GetResourceread and when Cloud Control applies the patch. The passing integ test shows this holds in practice for SQS; it is not proven to be deterministic across calls in general.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license