fix(jwks): read the public keys document under one name, and document rotation - #246
Merged
Conversation
… rotation Closes #175 and #243. validateEnvs.sh required JWKS_PUBLIC_KEYS, and the configuration reference and .env.example named only that, but token verification read SEAMLESS_JWKS_PUBLIC_KEYS. A deployment that set exactly what this API asked for started cleanly and then threw on every JWT verification, because getPublicKeyByKid could not find the secret. Reproduced against the built image with only the documented variable set. The managed instance is unaffected because its Terraform injects the same secret under both names, a workaround recorded in that repo rather than fixed here. Everything now uses SEAMLESS_JWKS_PUBLIC_KEYS. Anyone setting only the unprefixed name is already broken and does not know it; after this they fail to start with the variable named. Rotation is documented rather than built. The read path already satisfies the acceptance criteria: the document is a list, every key in it is published and can verify by kid, and only the active kid signs. What was missing was a written procedure, now in docs/production-operations.md as the three-step overlap, with a test that signs a token with the outgoing key and verifies it after the active kid has moved on. The server does not rotate its own keys, and will not. It has no secret-store write path, and environment variables are fixed for a process's lifetime, so a server that rotated could not observe the result without a restart it cannot trigger. This matches seamless-iac ADR 0011, which settles ownership by reading this repo's secretsStore. So the empty ensureKeys() production branch is removed, with initKeys, their specs and the dev-stack invocation. It advertised a capability that is not going to exist, and its development branch wrote a keypair to ./keys that nothing reads: signingKeyStore keeps dev keys under ./keys/dev and generates them lazily.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #175 and #243.
A deployment trap, reproduced
validateEnvs.shrequiredJWKS_PUBLIC_KEYS. The configuration reference,.env.exampleand the/.well-known/jwks.jsonhandler all named that too. But token verification readSEAMLESS_JWKS_PUBLIC_KEYS, andgetSecretthrows on a missing secret rather than returning empty.So a deployment that set exactly what this API documented started cleanly and then threw on every JWT verification. Reproduced against the built image with only the documented variable set:
The failure lands on the first authenticated request, not at boot, which is the worst place for it.
The managed instance was never affected: its Terraform injects the same secret under both names, with a comment explaining why, and
portal-auth/README.mddocuments it as "Injecting only the former passes startup but fails JWT verification at runtime." That is an API defect worked around in the infrastructure repo, so it never got fixed here.Everything now uses
SEAMLESS_JWKS_PUBLIC_KEYS.Breaking, deliberately. Anyone setting only the unprefixed name is already broken and does not know it. After this they fail to start, with the variable named, instead of serving an instance that cannot verify a token it just issued. Trading a silent runtime failure for a loud boot failure is the point.
#175, rescoped against ADR 0011
The issue asks the API to implement rotation. It should not, and I stopped to check rather than build it.
seamless-iacADR 0011 (Accepted, 2026-08-28) decides "Terraform owns the JWKS document. The runtime never writes it." It reaches that by reading this repo'ssecretsStore.tsand observing that runtime ownership would require a real secret store with write access, a task role granted write on a bundle it currently only reads, and a live fetch instead of environment variables, because a process cannot see its own new key without a restart it has no way to trigger. That repo's AGENTS.md adds: "Do not add a runtime write path without revisiting ADR 0011."The same ADR notes rotation needs no image change at all, because the runtime already handles a multi-key document. That matches what is here: the document is a list,
loadAllPublicKeyscaches every entry by kid, verification resolves by the token's ownkid, and only the active kid signs.So the acceptance criteria were already met by the read path plus Terraform. What was genuinely missing was the written procedure, and a test.
docs/production-operations.mdnow carries the three-step overlap (add, flip, retire), why the steps cannot be collapsed, why a restart is needed at each one, why key ids must be environment-variable safe, and a pointer to the automated procedure for managed instances.#243 falls out
The empty
ensureKeys()production branch was the placeholder reserved for the rotation that is not going to live here, sokeyManager.ts,initKeys.ts, their two specs, and the dev-stack invocation are removed. Its development branch wrote a keypair to./keysthat nothing has ever read:signingKeyStorekeeps dev keys under./keys/devand generates them lazily, and that is what the JWKS route serves.Tests
A new
keyRotationOverlap.spec.tsexercises the contract with real RSA keys and real jose, rather than asserting about mocks:refresh_token_ttlVerification
1216 tests passing, 1 skipped. Typecheck, lint, format clean.
sh -nand shellcheck clean onvalidateEnvs.sh. Coverage 98.93% statements, 96.45% branches.Follow-up for seamless-iac
JWKS_PUBLIC_KEYSis now unused by the API, so the duplicate injection ingrand-cross/main.tfandportal-authcan be dropped and the README note retired. Harmless until then. Worth doing in the same pass as the #205 migration task.