Skip to content

chore(purchase): remove the dead server-side SQS purchase path - #1812

Merged
cristim merged 15 commits into
mainfrom
chore/remove-dead-sqs-path
Nov 6, 2026
Merged

cristim merged 15 commits into
mainfrom
chore/remove-dead-sqs-path

Conversation

@cristim

@cristim cristim commented Nov 6, 2026

Copy link
Copy Markdown
Member

Summary

Closes #399 by removing the server-side SQS async purchase-execution path. Nothing ever produced these messages: there's no queue, no event source mapping and no SendMessage anywhere in the repo, its IaC or the sibling cli/go/mcp repos, and git log -S finds none in history. So ProcessMessage, its approve/cancel/execute-by-ID handlers, the SQS batch handler and SQS_REPORT_BATCH_ITEM_FAILURES were unreachable money-path code. That included a second, weaker approver gate and an execute-by-ID path that bypassed executableByScheduler's web-source rule for approved rows.

The Lambda now rejects an aws:sqs event as unrecognized (internal/server/lambda.go). It is no longer silently processed. If a stray mapping ever existed, Lambda would retry and then DLQ the batch: loud and bounded.

How (13 commits, each 400 lines or fewer; +509 -2028)

  1. Re-home every test that went through the SQS path to the direct paths, deleting the old test in the same commit (58c8038 to ddbd504). The direct paths are ApproveExecution, CancelExecution, the cron sweep, claimAndExecuteWithin, ApproveAndExecute and the API handlers. Each guard the SQS path enforced is now pinned on the path that remains:

    Guard Pinned by
    Token approve and cancel with TTL TestManager_ApproveExecution_ExpiredToken, CancelExecution_ExpiredToken, new TestValidateApprovalToken_TTLAndPrecedence
    4-eyes on the token path TestManager_ApproveExecution_FourEyesOn_TokenActorEqualsCreator_Denied and its allowed/unresolvable twins
    Approver contact-email gate (case-insensitive) new ApproverMatchIgnoresEmailCase
    Web-source and AutoPurchase=false rows never auto-run; gate refusals before the claim, including 4-eyes from feat(purchase): refuse a Redshift dispatch while another lineage of the tuple is open #1807 ProcessScheduledPurchases_WebSourceSkipped, TestScheduler_RefusesBeforeClaim, FourEyesOnSkipsAutoRow
    Redrive, retry and ladder guards armed_redrive, aws_no_token_redrive and ladder_execution_recheck, via the claim funnel
    Claim CAS against pause/resume the pause_claim integration suite
  2. Reject aws:sqs events, then remove the batch handler, the env var and the message handlers (782fce1 to 2f04055). The rejection tests landed first.

  3. Docs and comments (55dc44e, fe39d83). CHANGELOG, known-issues.md, and SQS-era wording in comments and test names. Assertions are unchanged.

The by-ID execute loader and its "approved rows bypass the gate" rule are deleted, not re-homed: nothing in production called them.

Behaviour change: an aws:sqs Lambda event now returns an error instead of being processed. SQS_REPORT_BATCH_ITEM_FAILURES is no longer read.

Verification (at fe39d83, rebased on main 803090d)

  • go build ./..., go vet ./internal/... and go vet -tags integration on purchase, api and server all pass.

  • go test -count=1 passes on ./internal/purchase/..., ./internal/api/..., ./internal/server/... and ./cmd/lambda/....

  • go test -count=1 -tags integration ./internal/purchase/... ./internal/api/... passes, with Postgres in testcontainers.

  • Mutation probes (each reverted):

    • Making the pause CAS (handler_purchases.go:304) also accept running fails TestPauseClaim_RunningWorkerCannotBeReclaimed and RejectsNonPendingStates. This ran before the rebase, at 722e0a92; the rebase didn't change that code.
    • Disabling the Redshift conflict refusal (redshift_lineage.go:127) fails all 3 re-homed Redshift gate integration tests.
    • The executor-side redrive check is killed by the purchase-package tests.
  • git grep -i 'sqs\|AsyncMessage\|ProcessMessage' only hits CHANGELOG and the tests that pin the aws:sqs rejection (outside docs/audits and lock and sum files).

  • Reviews:

    • the plan got 2 adversarial passes;
    • the full change got 2 adversarial passes, closed on agreement.

    Pass 1 caught redshift_gate_integration_test.go (added by feat(purchase): refuse a Redshift dispatch while another lineage of the tuple is open #1807 after the plan base) still calling the removed claimAndExecute. It's fixed in 2f04055: the tests now use claimAndExecuteWithin with notified rows.

Out of scope

internal/config/store_postgres_recommendations.go:639 mentions a "late redelivery" of collection triggers. That's unrelated to this path and untouched.

🤖 Generated with Claude Code

cristim and others added 13 commits November 6, 2026 15:58
…ge path

ApproveExecution now carries the plan-id propagation test, and the shared
token check gets its TTL and precedence table (wrong token on an expired
row reports invalid, not expired). The SQS-entry duplicates are removed
because each has a named twin in approvals_test.go that asserts the same
outcome:
- ApproveHappyPath: TestManager_ApproveExecution_Success (:83)
- FourEyesOn_SelfApproveDenied: ..._FourEyesOn_SQSActorEqualsCreator_Denied (:713)
- FourEyesOn_DifferentApproverSucceeds: ..._FourEyesOn_DifferentActor_Allowed (:757)
- CancelHappyPath: TestManager_CancelExecution (:851)
- ApproveRejectsTokenMismatch: ..._InvalidToken (:178)
- ExpiredTokenRejectedBeforeApproverLookup: ..._ExpiredToken (:1079) and
  TestManager_CancelExecution_ExpiredToken (:1284)

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The case and whitespace insensitive comparison was tested only through
the SQS approver check. It is now asserted on authorizeApprovalAction,
the production gate, for both sides of the comparison. The SQS-only
tests are removed: the missing-actor cases exist only because the SQS
message carried the actor (the HTTP twin is
TestHandler_approvePurchase_RejectsMissingContactEmail), the mismatch
and non-ASCII cases are covered by
TestHandler_authorizeApprovalAction_NonASCIILettersNotFolded and the
no-approver case by the "no per-account contact email" assertions.

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd claim funnel

Multi-account ack, redelivery and failure-notification tests now run
through ProcessScheduledPurchases (Executed counts the ack), and the
claim-pin, save-error and success tests call claimAndExecuteWithin
directly with notified rows. The SQS execute_purchase refusal table is
removed (every case is a row of TestScheduler_RefusesBeforeClaim), as are
the by-ID load error test, the approved-row gate bypass test and the
approved-row cap-skip test, which only exercised the SQS-only rules. A
funnel-level test keeps the ErrStepCommitmentCap assertion.

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…unnel

The armed Azure savings plan, AWS no-token and ladder recheck tests call
claimAndExecuteWithin on notified rows instead of an SQS message. The
shared harness plan now carries a step cap so the post-claim cap recheck
passes and the guards under test, not the cap, decide the outcome.

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The retry redrive and fan-out suites ran the persisted successor through
an SQS execute message. They now keep the pending status the retry
handler persists and approve it with ApproveAndExecute, the production
path, with the same purchase-token assertions.

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pause/claim worker is ProcessScheduledPurchases (a pending row is now
skipped silently instead of refused), ramp-step root rows run through the
sweep, per-account retry rows are pending and approved with
ApproveAndExecute, and the claim-race and retention tests call
claimAndExecuteWithin and claimAndRedrive.

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing in this repo produces an SQS event, so the async purchase path
behind it was dead. An aws:sqs event now falls through to the
"unrecognized Lambda event shape" error instead of running purchase
execution without the 4-eyes and contact gates the HTTP path enforces.
The new test and the converted detect, routing and cmd/lambda rows failed
before this change (the event routed and returned nil).

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With aws:sqs events no longer routed, handleLambdaSQSEvent and its
batch-failure helper are unreachable. Delete them with their tests; the
flag loader they used goes in the next commit.

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sage

Delete the flag, its loader and startup validation, Application
.HandleSQSMessage, PurchaseManagerInterface.ProcessMessage, the mock's
ProcessMessageFunc and their tests. The os and strconv imports in
lambda.go go with the loader.

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
handleApproveMessage, handleCancelMessage and their private approver gate
(verifyAsyncApprovalActor, loadAsyncExecutionForApproval,
matchActorAgainstApprovers, gatherApproverContactEmails) had no caller
outside ProcessMessage. The production approver gate is
api.Handler.authorizeApprovalAction. Trim the approve and cancel cases,
message types and the token/actor fields from the message envelope, and
drop the two missing-token subtests (twin: TestManager_ApproveExecution_EmptyToken
and the CancelExecution invalid-token test).

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Delete the rest of messages.go (ProcessMessage, handleExecutePurchase,
checkAutoExecuteGate, the message types) and its tests, plus the
claimAndExecute wrapper whose last callers were re-homed. The cron sweep
keeps claimAndExecuteWithin, isMultiAccountAckable and
executableByScheduler. The by-ID execute loader and its "approved rows
bypass the gate" rule had no production caller.

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reword comments that described the removed SQS path, add the CHANGELOG
Removed entry (the Lambda rejects aws:sqs events and
SQS_REPORT_BATCH_ITEM_FAILURES is no longer read), and note the now
unreachable "approved" entry of the claim funnel's from-set in
known-issues.md as a follow-up.

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cristim cristim added priority/p2 Backlog-worthy severity/medium Moderate harm type/chore Maintenance / non-user-visible triaged Item has been triaged urgency/this-quarter Within the quarter impact/few Limited audience effort/m Days labels Nov 6, 2026
@coderabbitai

coderabbitai Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 1 minute.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 0a59d3a and 7cfc88c.

Files selected for processing: 44

  • CHANGELOG.md
  • cmd/lambda/main_test.go
  • internal/api/handler_history.go
  • internal/api/handler_purchases_pause_claim_integration_test.go
  • internal/api/handler_purchases_retry_fanout_test.go
  • internal/api/handler_purchases_retry_redrive_test.go
  • internal/api/handler_purchases_test.go
  • internal/api/types.go
  • internal/config/store_postgres_no_submission_integration_test.go
  • internal/purchase/approvals.go
  • internal/purchase/approvals_test.go
  • internal/purchase/armed_redrive_test.go
  • internal/purchase/aws_no_token_redrive_test.go
  • internal/purchase/cancel_scheduled_integration_test.go
  • internal/purchase/claim.go
  • internal/purchase/claim_pin_test.go
  • internal/purchase/claimed_row_check_test.go
  • internal/purchase/coverage_extra_test.go
  • internal/purchase/due_age_cap_test.go
  • internal/purchase/execution.go
  • internal/purchase/failure_notify_test.go
  • internal/purchase/failure_notify_writers_test.go
  • internal/purchase/ladder_execution_recheck_test.go
  • internal/purchase/manager.go
  • internal/purchase/manager_test.go
  • internal/purchase/messages.go
  • internal/purchase/messages_test.go
  • internal/purchase/messages_token_ttl_test.go
  • internal/purchase/money_path_regression_test.go
  • internal/purchase/plan_step_atomic_integration_test.go
  • internal/purchase/plan_step_retention_integration_test.go
  • internal/purchase/ramp_step_progress_integration_test.go
  • internal/purchase/redshift_gate_integration_test.go
  • internal/purchase/single_claim_integration_test.go
  • internal/purchase/step_cap_test.go
  • internal/server/app.go
  • internal/server/app_test.go
  • internal/server/handler.go
  • internal/server/handler_test.go
  • internal/server/interfaces.go
  • internal/server/lambda.go
  • internal/server/lambda_test.go
  • internal/testutil/mocks.go
  • known-issues.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

cristim and others added 2 commits November 6, 2026 20:12
…ecuteWithin

Main added three integration tests that call claimAndExecute, which this branch
removes. Run them through claimAndExecuteWithin on notified rows, like the rest
of the file.

Refs #399

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cristim

cristim commented Nov 6, 2026

Copy link
Copy Markdown
Member Author

Update after opening: main gained 0a59d3a (#1809, a Redshift adoption-window refusal), which adds three integration tests that call claimAndExecute, the function this PR removes. The merged integration build would have failed.

  • Merge, not rebase: to avoid force-pushing an open PR, I merged main (7befa43) instead of rebasing.
  • 7cfc88c: runs the three new tests through claimAndExecuteWithin on notified rows, like the rest of redshift_gate_integration_test.go. Their assertions are unchanged.

Proof at 7cfc88c:

  • go build ./... and go vet -tags integration on purchase, api and server pass.
  • Unit tests pass on purchase and server.
  • go test -tags integration -run Redshift ./internal/purchase/ passes.
  • Probe: with if false && redshiftFinishedStatus(...) at redshift_lineage.go:132, TestRedshiftDispatchGate_RecentCompletedBuyRefusedBeforeSubmission fails (exit 1). So main's new refusal is still reached through the remaining path. The file was restored afterwards.

@cristim

cristim commented Nov 6, 2026

Copy link
Copy Markdown
Member Author

APPROVED 7cfc88c by claude-ui-fix-1

Cold review at this SHA, against origin/main 0a59d3a. The lineage gates from #1807 and #1810 are in the base. In production code this PR only removes code and changes comments. It removes:

  • messages.go, including ProcessMessage, the execute/approve/cancel handlers, verifyAsyncApprovalActor and checkAutoExecuteGate;
  • the SQS branch in lambda.go, and HandleSQSMessage;
  • SQS_REPORT_BATCH_ITEM_FAILURES;
  • the claimAndExecute wrapper, and its mock method.

No guard logic changes.

Findings

  1. Low, non-blocking. At internal/purchase/manager.go:217, the sweep CAS still claims from approved, but no caller can reach that state now:

    • Its only remaining caller, processOneExecution (:1009), is fed by GetPendingExecutions, which selects only pending and notified (internal/config/store_postgres.go:1897).
    • The approve CAS goes straight to running (approvals.go:432), and stranded approved rows use claimRunning([approved]) (:677).

    This keeps a money-path CAS wider than it needs to be. As a deletion probe, dropping approved fails only TestClaimAndExecuteRefusesArmedAzureSavingsPlanRetry, whose fixture uses an approved row. The PR notes this in known-issues.md without an issue number. Suggest a follow-up issue linked there, then drop approved and move that fixture to notified.

  2. Info. docs/audits/codebase-audit-2026-09-02.md still describes the SQS path. It is a dated audit, so leaving it is reasonable.

Every entry point reaches executeAndFinalize (manager.go:316), which runs preExecutionRefusal (:323) and then stampRedshiftDispatch (:325):

  • Approvals: token approve (handler_purchases.go:1177), session approve (:1293), direct execute (:3997) and run-now (approvals.go:379) all go through transitionApproveAndExecute. That applies four-eyes (approvals.go:421), the CAS to running (:432), then runClaimedAndMint (:479). Retry successors use the same path.
  • Cron sweep: executableByScheduler (manager.go:989; web source, AutoPurchase and four-eyes), then claimAndExecuteWithin (:1009; claim pin, age and step caps), then executeAndFinalize.
  • Scheduled fire: claimRunning on scheduled (scheduled_fire.go:132), then the age cap, then executeAndFinalize (:163).
  • Stranded or armed re-drive: claimAndRedrive (manager.go:677), then executeAndFinalize.
  • Cancel: CancelExecution (handler_purchases.go:1839), unchanged.
  • Guards that lived in the removed code:
    • checkAutoExecuteGate duplicated executableByScheduler, which is still the sweep gate.
    • The approver contact-email check lives in the HTTP authorizeApprovalAction, with case-insensitive and non-ASCII tests at handler_purchases_test.go:7044.
  • SQS events now: an aws:sqs payload matches neither the HTTP nor the scheduled shape, so it is rejected. TestHandleLambdaEvent_SQSEventIsUnrecognized and TestHandler_SQSEventIsRejected cover that.

Repointed Redshift tests (redshift_gate_integration_test.go:98, :161, :201, :223, :239) now call claimAndExecuteWithin on notified rows. That is the production sweep funnel with the state the sweep feeds it, not a test-only shortcut, and it also exercises the step-cap recheck.

Nothing left on SQS. I grepped Go, config, env names, terraform, cloudformation, arm, iac, scripts, .github and the frontend. The only hits are the CHANGELOG, the rejection tests, the audit doc and lockfile hashes. No IaC resource or IAM sqs: grant existed before this PR or exists now.

Mutation probes (scratch copy, each reverted):

  • Removing the four-eyes call fails TestManager_ApproveAndExecute_FourEyesOn_DeniesSelfApprove.
  • Treating a lost claim CAS as a win fails TestScheduledSweepOverlapDoesNotDoubleExecute (2 purchases, expected 1) and TestClaimPin_AttachBetweenGateAndClaimIsRefused.
  • Ignoring or removing stampRedshiftDispatch fails all 5 Redshift gate integration tests, including the 3 repointed ones.
  • Removing preExecutionRefusal fails 8 tests, including TestLadderRecheck_EveryExecutorEntryRefuses and TestRunNow_NotifiedRowReachesArmedRedriveGuard.

Results (macOS, throttled):

  • go build ./... passes.
  • go vet passes, with and without -tags integration.
  • go test on purchase, api, server, cmd/lambda and testutil passes.
  • -tags integration for the full purchase and api packages passes.
  • CI was still pending on several jobs when I checked, so the merge needs green CI at this SHA.

Gaps:

  • No real-account check (none is needed for a removal).
  • An SQS event-source mapping created outside this repo can't be ruled out from source. If one exists, its batches now fail loudly.

@cristim

cristim commented Nov 6, 2026

Copy link
Copy Markdown
Member Author

APPROVED 7cfc88c by claude-qa-ui-4

Cold review. Outside the deletions, the production changes are comments and the removal of the claimAndExecute wrapper (internal/purchase/manager.go:203-213). No IaC is touched.

Guards per path at this head (unchanged from main):

Path Entry Claim Four-eyes Redshift gate
Session approve internal/api/handler_purchases.go:1293 → approvals.go:347 claimAndCheck enforceFourEyesPolicy (approvals.go:421) executeAndFinalize (:479 → manager.go:325)
Token approve, Run now, DirectExecute handler_purchases.go:1177 / :458 / :3997 same funnel same same
Cron/due sweep manager.go:1036 → :989/:1009 TransitionExecutionStatusPinned (:217) executableByScheduler (:916) manager.go:235 → :325
Delayed fire scheduled_fire.go:127 claimRunning(scheduled) at schedule time :163
Stranded re-drive manager.go:671 claimRunning(approved) approved via the approve funnel only :696

Retry successors, plan-step attach and the ladder execute through those paths. Cancel is a token-pinned conditional update.

  • The deleted checkAutoExecuteGate only called executableByScheduler for pending/notified rows, which the cron path does before its claim. Its other logic was a bypass for approved rows (now gone) and the async approver-email check; the HTTP path keeps authorizeApprovalAction.
  • The 3 repointed Redshift tests call claimAndExecuteWithin on notified rows: the real CAS, then claimedRowRefusal, then executeAndFinalize, then stampRedshiftDispatch, with no shim.
  • No SQS producer, consumer, IAM action, queue URL, env var, IaC resource or config field remains. An aws:sqs event now gets "unknown" from detectLambdaEventType (internal/server/lambda.go:48-77) and HandleLambdaEvent returns an error (:41-45), so it fails loud. The repo never had a queue in IaC, so there is no deploy-order concern.

Runs (GOMAXPROCS=4, -p 2):

  • Build and vet (with and without the integration tag) pass; unit tests pass.
  • Integration purchase, api (scoped) and config pass.
  • golangci-lint: one prealloc in store_postgres_history_page_integration_test.go:141, an untouched file.
  • Probes, each killed:
    • Redshift gate call removed (fails ConflictRollsBackStamp, TwoLineagesRaceOnlyOneBuys, RootAndAccountSuccessorsOfOneFamilyDoNotBothBuy and RecentCompletedBuyRefusedBeforeSubmission);
    • four-eyes removed from executableByScheduler;
    • enforceFourEyesPolicy disabled;
    • the CAS status = ANY($3) condition relaxed.

@cristim
cristim merged commit c318a6b into main Nov 6, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/m Days impact/few Limited audience priority/p2 Backlog-worthy severity/medium Moderate harm triaged Item has been triaged type/chore Maintenance / non-user-visible urgency/this-quarter Within the quarter

Projects

None yet

Development

Successfully merging this pull request may close these issues.

chore: provision (or remove) the SQS queue and Lambda event source mapping for async purchase execution

1 participant