[cristim]

Closes #702 (Redshift re-drive idempotency). This is PR3b, the last of the series. Earlier pieces, already merged:

• fix(purchase): persist the Redshift first-dispatch time and pass it as RequestedAt #1762: the first dispatch time is stamped and passed as RequestedAt.
• feat(purchase): refuse a Redshift dispatch while another lineage of the tuple is open #1807: one lineage at a time per Redshift tuple.
• fix(purchase): refuse a Redshift dispatch inside the adoption window #1810: the adoption-window refusal.
• feat(api): gate Redshift retries and copy the first dispatch time #1816: the Retry gate and the stamp copy.
• feat(purchase): record Redshift purchase attempts in a durable journal #1824: the attempt journal.

This PR lifts the guard that refused every Redshift re-drive and Retry.

Why

Redshift reserved nodes have no AWS idempotency token, so re-driving a Redshift purchase could buy a second node. The earlier PRs built the safety net:

• a stable RequestedAt per lineage, so go's dedupe can find the original node;
• a per-tuple dispatch gate;
• an adoption-window refusal;
• a Retry gate with a 10-minute settle wait and permanent refusals;
• a durable attempt journal.

With all of that in place, Redshift retries can be allowed.

What (9 commits, each under 400 lines)

1. Writer scan fix (test). The idempotency-key writer scan now cuts UPDATE and upsert SET text at a depth-0 WHERE, so an assignment after a subselect is still seen.
2. ResolveOpsHint export (refactor). It moves into internal/purchase so the dispatch gate can use the persistent-failure hints.
3. Failed rows as open lineages. A failed Redshift row that may have bought, and can still be retried, now blocks a second lineage of the same tuple.
   • These rows are excluded, because Retry refuses them for good: linked, settled (when the settlement schema exists), stamp-less, past 24h, never submitted at attempt 0, or with a persistent hint.
   • When only failed rows conflict, the fixed refusal text says to retry or settle that purchase.
4. The lift. The AWS no-token refusal now covers EC2 and Compute only. Redshift retries and re-drives go through the Retry gate and the lineage gate. The old reason text stays allowlisted for rows stored before the lift. A failed row whose Retry the remaining guard still refuses no longer blocks a new lineage.
5. Reaper wording. A reaped Redshift row is no longer described as safe to retry straight away.
6. Fail-closed journal guard. A Redshift dispatch at attempt > 0 is refused, with a plain error, before stamping or calling the provider when the Manager has no attempt journal. Owner decision 2026-11-06b, Q2 = B.
7. History and openapi.
   • History marks a Redshift row redrive-unsafe when Retry would refuse it for good. This is a presentation hint on the app clock; the gate decides on the database clock.
   • The 409 for a retry inside the settle wait documents retry_after_seconds and Retry-After.
   • The History page tells the user how long to wait.
8. End-to-end test. Redshift retries and stranded re-drives run through the Manager with an injected journal, and a fake client applies go's RequestedAt - 5m adoption window.
9. Known issues. known_issues/39 is updated for the lifted state.

Evidence

• Plan: two adversarial plan-review passes before any code.
• Code review: two non-author passes. Pass 1 found three Lows, all fixed: the failed-conflict text, dropping failed rows that the remaining guard refuses, and the end-to-end stamp coalescing. Pass 2 had no findings.
• Local verification (rebased tip): build, unit tests for purchase, config, api and mocks, and the config and purchase integration tests all pass. Pre-commit is clean over the range, and every commit passed its own hooks.
• Mutation probes: each of these makes a test fail:
  • a scan that cuts at any WHERE depth;
  • no hint filter;
  • no linked exclusion;
  • no settled exclusion;
  • Redshift back in the guard;
  • the reaper calling a row safe;
  • no journal guard;
  • History always passing a nil stamp;
  • History treating a missing stamp as fresh;
  • no no-stamp refusal;
  • no never-submitted exemption.
• Not run locally: the frontend test (the CI frontend job runs it).

Deploy

Deploy PR3b to each environment no earlier than 24h after migration 000161 and #1824 are live there. A Redshift row dispatched before the attempt journal went live in an environment carries no journal record. If AWS changes that tuple's offering within the 24h re-drive horizon, a retry of such a row could buy twice. Waiting 24h after the journal goes live means every retryable Redshift row was dispatched with the journal (reviewer finding from the #1832 review). Dev: #1824 went live at 2026-11-06T23:38:42Z, so this PR merges no earlier than 2026-11-07T23:39Z. Prod ordering is tracked in a follow-up issue.

Merge only after #1824 is deployed with its migration (000161), so the production Manager has the journal. Otherwise C10b refuses every Redshift retry, which fails closed. There are no new migrations.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 47 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: LeanerCloud/cloud-commitments-platform
• Branch: feat/702-redshift-redrive
• File: .coderabbit.yaml
• Commit: 7e89c48
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Advanced
• Run ID: c49b6a0d-24ec-4317-9be1-910990c3a6c9

Commits

Reviewing files that changed from the base of the PR and between 71b5e29 and 7e89c48.

Files selected for processing: 31

• frontend/src/__tests__/history-retry-button.test.ts
• frontend/src/history.ts
• internal/api/handler_history.go
• internal/api/handler_history_test.go
• internal/api/handler_purchases.go
• internal/api/handler_purchases_redshift_retry_test.go
• internal/api/handler_purchases_retry_redrive_test.go
• internal/api/handler_purchases_test.go
• internal/api/openapi.yaml
• internal/config/store_postgres_redshift_lineage.go
• internal/config/store_postgres_redshift_lineage_integration_test.go
• internal/config/store_postgres_redshift_lineage_test.go
• internal/purchase/aws_no_token_redrive_test.go
• internal/purchase/errnotes.go
• internal/purchase/errnotes_test.go
• internal/purchase/execution.go
• internal/purchase/execution_skipped_accounts_test.go
• internal/purchase/idempotency_key_writers_test.go
• internal/purchase/manager.go
• internal/purchase/ops_hint.go
• internal/purchase/ops_hint_test.go
• internal/purchase/plan_step_proofs_test.go
• internal/purchase/reaper.go
• internal/purchase/reaper_test.go
• internal/purchase/redshift_gate_test.go
• internal/purchase/redshift_journal_guard_test.go
• internal/purchase/redshift_lineage.go
• internal/purchase/redshift_redrive_e2e_test.go
• internal/purchase/requested_at_test.go
• internal/purchase/stranded_redrive.go
• known_issues/39_redshift_dedupe_window.md

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

[cristim]

CHANGES REQUESTED 7e89c48 by claude-qa-ui-4

Blocking:

1. Two same-family successors of one token can dispatch inside the settle window. The settle wait is checked only when a Retry is created, against that row's own predecessor (internal/api/handler_purchases_redshift_retry.go:28 → internal/purchase/redshift_retry.go:51). At dispatch, a same-family row conflicts only while running (internal/config/store_postgres_redshift_lineage.go:115), and the failed clause matches other families only (<> $2, :67-71).
   • The reachable sequence: a fan-out root R0 and its child X0 both end failed with an unknown outcome. Retrying R0 gives R1, whose fan-out child X' gets the key <root>:<acct>. Retrying X0 gives X1 with the same key and the same copied FirstDispatchAt (handler_purchases.go:2694-2750). The submit dedupe (2 min, same creator; :3673, :2845) does not apply.
   • R1 is approved, X' sends, the response is lost and X' ends failed. X1 is then approved: the gate passes, because X' is failed and in the same family.
   • go's journal skips a prior attempt from another execution on the same offering (dedupe.go:229-245), and the window lookup misses X''s node while DescribeReservedNodes lags. X1 buys a second node.
   • Before this PR, armedRedriveRefusal closed this by refusing every Redshift attempt above 0.
   • Suggested fix: in the conflict query, let same-family failed rows that may have sent (not no_submission AND retry_attempt_n = 0) conflict while updated_at > now() - settle on the DB clock, plus an integration test for a root successor and a child successor of one token.

Checked and holding:

• Gate order on dispatch: armedRedriveRefusal, then the nil-journal guard (internal/purchase/manager.go:421-433, refusing attempt > 0 with no journal, so the pre-DB Manager refuses), then the lock, the stamp and horizon, and the conflicts (redshift_lineage.go:108-133), then the provider.
• The lift is Redshift only: recRedriveRefusalReason and awsPurchaseIsEC2 (manager.go:658-700) still refuse EC2, Azure savings plans and unknown providers. TestAWSRedriveRefusalSetIsExact pins the set.
• The journal is passed on every path (execution.go:834). A successor on a different offering after an unrecorded send is refused by go (dedupe.go:239).
• The stranded re-drive only re-drives approved rows that were never claimed, so never sent. The reaper only fails rows. The stamp is copied on retry.
• Owner decisions Q1-Q8 are followed. Q1 and Q4 are in known_issues/39; Q6 is follow-up Redshift: two identical-tuple recommendations in one execution buy one node #1809; Q8 is go#700.
• History: applyRedshiftRetryHint runs only for Redshift. openapi documents the 409, Retry-After and redrive_unsafe. The errnotes allowlist has both new texts and keeps the old EC2 text (errnotes.go:41-42, :123). The non-Redshift paths are unchanged.

Runs (GOMAXPROCS=4, -p 2):

• Build and vet pass; unit purchase/config/api/mocks pass.
• Integration purchase and config pass; api scoped to Retry|Redrive|Redshift|History|Purchase passes.
• CI run 42314561416 at this head is green.
• Probes, 10 of 10 killed:
  • nil-journal guard removed, or moved to attempt >= 2;
  • settle 0;
  • API settle check removed;
  • journal not passed;
  • lift widened to EC2;
  • same-tuple refusal skipped for successors;
  • stamp copy dropped;
  • failed clause off;
  • dropPersistentFailures bypassed.
• No test covers the sequence in finding 1.

Non-blocking:

• golangci-lint: one prealloc in store_postgres_history_page_integration_test.go:141, an untouched file (CI Lint passes).

[cristim]

APPROVED 7e89c48 by claude-darling

Cold review; I did not write this change. This is a money path: it lifts the Redshift redrive guard. I found no double-buy path once the journal is in place. One low finding needs a deploy-order note before this reaches an environment where the journal has been live for less than 24h (see the end).

Owner decisions (2026-11-06b)

Q	Lands at
Q1 A, the 24h block	known_issues/39, "A failed direct-purchase row cannot be settled…"
Q2 B, journal plus fail-closed	app.go:875 (AttemptJournal: pgStore); execution.go:834 passes it per rec; manager.go:431 refuses hasRedshiftRec && RetryAttemptN>0 && attemptJournal==nil
Q3 A, the wording	ResolveOpsHint (ops_hint.go), handler_history.go:432-447, openapi.yaml:1471-1501,7716, history.ts:1650
Q4 A	known_issues/39, the no_submission dead-end bullet
Q5-Q8	delivered by PR2b and #1810; Q8 non-blocking

Double-buy paths

1. Settle period. Measured on the DB clock (store_postgres_redshift_retry.go:27), with a 10m wait (redshift_retry.go:11, >= at :52). The dispatch gate re-checks the 24h horizon on the DB clock (store_postgres_first_dispatch.go:40, redshift_lineage.go:121). updated_at comes from the post-send failure write, or the reaper flip at 10m or more, which is past the 3m30 budget. The go contract (dedupe.go:46-48) leaves the listing-lag wait to the caller, and payment-pending nodes count as owned (client.go:584).
2. Offering-ID change after an unrecorded send. The token is DeriveIdempotencyToken(lineageKey, i) (execution.go:833), and the key and rec order are copied to the retry (handler_purchases.go:2694), so the token is stable. recordSending must succeed before the send (dedupe.go:251-263). The successor sees the predecessor's sending and refuses (dedupe.go:239), and a purchased record whose node isn't listed also refuses (:217). Closed, except for the pre-journal rows below.
3. fix(purchase): refuse a Redshift dispatch inside the adoption window #1810's same-tuple refusal runs on every attempt (redshift_lineage.go:129-133). The new failed-row clause (store_postgres_redshift_lineage.go:61-72) only drops rows Retry can never redrive, so the worst case is the known under-buy.
4. feat(api): gate Redshift retries and copy the first dispatch time #1816's PR3a gates. checkRedshiftRetryGate runs after the eligibility gates, and force can't override it (handler_purchases.go:2516-2521). The executor runs armed, ladder, journal, stamp and horizon, then the conflict check (manager.go:340-343,420-434).
5. Nil journal. There are only two constructions. The pre-DB manager (app.go:551) has a nil ConfigStore and is replaced before any claim (app.go:685,992), and there are no other NewManager callers. TestRedshiftRetryDispatch_NilJournalRefused pins the guard.
6. Concurrency. The link CAS (handler_purchases.go:2771-2780) rolls back a second retry. Execution is single-claim (manager.go:222). The tuple advisory lock plus READ COMMITTED serialize the gates. A stranded redrive only claims never-sent approved rows.

History, openapi, wording. Both new stored notes and the legacy EC2+Redshift text are allowlisted with QuoteMeta (errnotes.go:41-42,123), and closure is tested with id-appended variants (errnotes_test.go:214). openapi documents retry_after_seconds and Retry-After. The reaper no longer says "safe to retry" for Redshift (reaper.go:278).

Non-Redshift unchanged. EC2 is still refused (manager.go:688-701), ResolveOpsHint is a byte-for-byte move, and every new branch is gated on Redshift. TestNonRedshiftRetryDispatch_NilJournalUnaffected and TestCheckRedshiftRetryGate_NonRedshiftNeverReadsClock cover it.

Finding: deploy order (LOW, but a real double-buy window).

• Affected rows: a Redshift row stamped after fix(purchase): persist the Redshift first-dispatch time and pass it as RequestedAt #1762 but dispatched before the journal (000161 plus feat(purchase): record Redshift purchase attempts in a durable journal #1824) was live has first_dispatch_at and no purchase_attempts record.
• Repro: retry such a row within its 24h horizon, after the 10m wait, while AWS has changed the offering ID. PriorAttempts is empty, the window lookup under the new offering misses the old node (dedupe.go:81), and a second node is bought.
• Likelihood: it needs a rare offering change. On dev the journal has been live since 71b5e29, so the window closes 24h after that.
• Ask: before merging, add a release note or a known_issues bullet: "deploy PR3b to each environment at least 24h after the journal (000161 plus feat(purchase): record Redshift purchase attempts in a durable journal #1824) is live there". Prod must follow that order.

Probes (scoped, each reverted, no survivors). Each of these fails at least one named test:

• nil-journal guard off (NilJournalRefused);
• settle skipped (3 tests);
• settle on the app clock (2 tests);
• no conflict check on redrive (3 tests);
• redrive skipping the horizon (PastHorizonRefusedAtEveryAttempt);
• the Retry gate skipped (3 tests);
• the journal dropped on redrive (3 tests).

Scoped -tags integration runs on purchase, config and api pass, and the config Redshift tests really ran against Postgres.

CI at this head: success, including Unit Tests and Integration Tests for api, auth, config, migrations, purchase and server. Run https://github.com/LeanerCloud/cloud-commitments-platform/actions/runs/42314561416.