Skip to content

fix(purchase): Redshift reserved-node purchases without tag-based idempotency #702

Closed

Description

@cristim

Summary

The member-account IAM templates do not grant ec2:CreateTags (and, for Redshift, Describe/CreateTags). Outside the host account the purchase idempotency tag is therefore never written, and the failure is only logged, so the pre-buy tag guard cannot detect a prior purchase.

Where

At commit b50ca38:

  • iac/federation/aws-target/terraform/main.tf:62-75 (plus the CloudFormation and cross-account variants of the same template)
  • Swallowed failure in cloud-commitments-go ec2/client.go:203-205

Failure scenario

A purchase runs against a member account assumed through the federation role. The EC2 or Redshift buy succeeds, the CreateTags call is denied, and the error is only logged. A later retry or re-drive checks the tag guard, finds no tag, and buys again. Found by source reading; not reproduced live.

Suggested fix

Add a scoped ec2:CreateTags (and the Redshift Describe/CreateTags actions) to every target template. Surface a tag-write failure on the execution record instead of only logging it.

Severity

High: a double purchase of a reserved instance is real money, and the guard that is meant to prevent it silently does nothing in member accounts.

Related: #157 (deploy-role grant gaps only, not the member-account purchase role).

Reviewer ID: MON-03 (money H3)
Found by the 2026-10-08 review fleet.

Activity

  1. cristim commented on Oct 9, 2026

    @cristim
    MemberAuthor

    PR 773 covers the IaC half: scoped ec2:CreateTags and redshift tag actions for all member-account purchase templates. Not in that PR, so this issue stays open: (1) surfacing a tag-write failure on the execution record (the swallow is in cloud-commitments-go ec2/client.go:203-205 plus platform handling); (2) the host runtime role has the scoped ec2:CreateTags already but no redshift:CreateTags/DescribeTags, so host-account Redshift buys are untagged. Existing member accounts need their template re-applied.

  2. cristim commented on Oct 9, 2026

    @cristim
    MemberAuthor

    Follow-up filed: #782 (the iacfiles IAM scoping test only inspects Terraform statements that have a Sid, so a Sid-less ec2:CreateTags on * would pass; found in the coordinator's delta review of PR 773).

  3. cristim commented on Oct 9, 2026

    @cristim
    MemberAuthor

    The EC2 half is merged at 7e2a3d8 (PR #773). The Redshift half stays open, pending owner decision 29 (sandbox tag test). Follow-ups: #779, #782, #42. Not closing this issue.

  4. IAmJSD commented on Oct 16, 2026

    @IAmJSD
    Collaborator

    Released: EC2 part already fixed by #773; the rest is blocked on decision 29 or needs a library change (details below)

  5. self-assigned this
    on Oct 16, 2026
  6. 15 remaining items

  7. cristim commented on Nov 4, 2026

    @cristim
    MemberAuthor

    PR1 of 4 is up: #1762 (first-dispatch stamp passed as RequestedAt; re-drive still refused).

  8. cristim commented on Nov 5, 2026

    @cristim
    MemberAuthor

    PR1 of 4 merged: #1762 (d4d35f4). It adds a purchase_executions.first_dispatch_at stamp, written before a Redshift buy and passed to the library as RequestedAt, as migration 000154. Redshift re-drive and retry are still refused. Prod: apply 000154 before the prod code deploy. PR2 (the cross-lineage conflict check) is in progress.

  9. cristim commented on Nov 5, 2026

    @cristim
    MemberAuthor

    Claimed by claude-fleet: PR1 merged (#1762). PR2, the cross-lineage conflict check, is local commits on fix/702-redshift-lineage-conflict:

    • conflict query and its matrix tests: committed;
    • dispatch gate: in its second review pass.

    PR2 opens after the gate commits and the branch is rebased onto main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions