[ThomasK33]

Summary

A CoderProvisioner now deletes the keys that status.pendingKeys records at earlier targets even while the current target fails its operator-token or entitlement checks. Each key is deleted by its recorded ID, at its recorded Coder URL, with the operator token of its own UID-bound control plane. A failed credential check still sends nothing to Coder. Entries with unpublished: true and the key in the Secret still wait for the current target.

Fixes #416

Background

The user goal for this run: fix the follow-up issues from the last run and land them on main.

Since #413, the cleanup of earlier-target keys ran only after the current target passed every check (control plane fetch, credential checks, operator token, entitlement). While the current target failed one of them, every retry returned before the cleanup, so a key at an older endpoint stayed valid in Coder. #416 records why #413 left it, and proposes this direction.

Implementation

All changes are in internal/controller/coderprovisioner_controller.go.

1. revokeEarlierTargetKeys runs right after the credential checks and before the operator-token and entitlement checks of the current target. It tries every status.pendingKeys record outside the target slot that is not in use. A record is in use when it is unpublished: true or names status.provisionerKeyID. Deletes go through the existing revokeKey, so 🤖 CoderProvisioner: bind each key record's operator token to the control plane UID #417/🤖 fix: a CoderProvisioner replaces only keys it recorded and binds key records to the control plane UID #420 token binding and 🤖 fix: a CoderProvisioner keeps every key record with IDs until Coder confirms its key gone #425 record retention are unchanged. When it drops a record, it writes status at once.
2. The retry rate is limited per provisioner UID with an in-memory sync.Map. After a try leaves a record that Coder did not confirm gone, the next try waits provisionerPendingKeyRetryInterval (2 minutes, the existing pending-key requeue). The operator-token wait requeues every 15 s, so without this a dead old endpoint got a request every 15 s.
3. The replacement pass (only when the key is replaced) now tries only records in use and the newly replaced record. It no longer repeats the early pass in the same reconcile. Without a replacement, no second pass runs.
4. Condition reports are built per record (keptKeyReport). Records that a pass holds back are reported without a Coder request: a changed control plane from the Kubernetes read, an ID-less record with the note "the operator checks Coder for it at the next key replacement". The replacement pass merges its reports with the early pass reports, so neither pass hides the other's entries. While the throttle runs, the merge uses the reports of the last try, so a replacement that drops a reported record also drops its report.
5. The KeyNameInUse refusal path loses its own cleanup call, because the early pass already ran in the same reconcile.

docs/tutorials/deploy-coderprovisioner.md describes the new behavior.

Validation

• New envtest TestCoderProvisionerKey_EarlierTargetRevokedWhileTargetNotReady (key A at control plane A, rotation to B whose delete of A failed, then a move to C):
  • C not entitled, and C operator token missing: key A is deleted at A with A's token, C gets no request, key B in the Secret stays. On main (commit b402b4b alone) these fail with key A must be deleted at A with the operator token of A (actual: no deletes).
  • Delete of A keeps failing: one try per interval, a second try after the interval. With C ready, the rotation does not try A a second time. On main the not-ready case fails with 0 tries.
  • Unpublished key at B (Secret write failed at B): nothing deletes B while C is not ready.
  • Credentials refused (unlabeled extraEnv Secret): no request reaches Coder and the record of A stays.
• New envtest TestCoderProvisionerKey_EarlierTargetReportsBesideUnpublishedKey: beside a held-back unpublished record, ControlPlaneChangedKeyKept names a recreated control plane (published and unpublished), and UnverifiedKeyKept drops an ID-less record once Coder has no key with its name.
• Negative controls (reverted afterward): dropping record.Unpublished from the in-use rule fails the unpublished case and TestCoderProvisionerKey_RevertAfterFailedRotation. Removing the throttle store fails the retry case. Letting the replacement pass retry early-pass records fails the ready-target case. Skipping the held-back control plane check fails the report test.
• TestCoderProvisionerKey_IDLessRecordKeepsOtherProvisionersKey now elapses the retry interval (ElapseEarlierKeyRetryInterval in export_test.go) before it expects a retry right after a failed lookup.
• Local: make verify-vendor, make test, make build, make lint, make test-integration, make docs-check.
• Two independent pre-PR reviews (gpt-6.1-sol, claude-opus-5-5). Fixed from them: the replacement pass bypassed the throttle and repeated tries; one held-back record froze every condition update; early drops could be lost on error returns; the key ID comparison now parses UUIDs; missing test for the fail-static rule.

Risks

• The throttle is in memory. An operator restart retries at once, which matches main.
• During a replacement, the records in use (the replaced key and unpublished keys) are tried as on main, without the throttle. A rotation that keeps failing retries them with controller-runtime's error backoff, as before.
• While the throttle runs, the reports of the last early try are reused. A record that joins the early pass's set in that window (for example a record that a rotation just published) gets its report with the next try, within 2 minutes.

---

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-31T00:39:17.883076Z	b0cb200	Manual request
🔒 Security Review	✅ Completed 2026-10-31T00:42:02.381931Z	b0cb200	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1554c176d1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: b0cb20057e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[ThomasK33]

@codex security review

[chatgpt-codex-connector]

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: b0cb20057e

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

• You comment "@codex security review"
• A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.