Skip to content

feat(operator): fetch JWT-SVID via go-spiffe SDK, remove spiffe-helper sidecar - #478

Closed
Alan-Cha wants to merge 12 commits into
mainfrom
feat/spiffe-sdk-jwt
Closed

feat(operator): fetch JWT-SVID via go-spiffe SDK, remove spiffe-helper sidecar#478
Alan-Cha wants to merge 12 commits into
mainfrom
feat/spiffe-sdk-jwt

Conversation

@Alan-Cha

@Alan-Cha Alan-Cha commented Jul 13, 2026

Copy link
Copy Markdown
Member

Summary

Closes #477. Replaces the standalone spiffe-helper sidecar container with a direct call to the SPIRE Workload API using the go-spiffe SDK — the same approach authbridge-proxy (the default agent sidecar mode) already uses.

Background

PR #473 added a spiffe-helper sidecar to the operator pod. The sidecar wrote a JWT-SVID to /opt/jwt_svid.token which the operator binary then read from disk. This approach had two problems:

  1. Abandoned image: ghcr.io/kagenti/kagenti-extensions/spiffe-helper is not published by the kagenti-extensions automated pipeline. The standalone image was removed from the pipeline in April 2026 when spiffe-helper was bundled into the combined authbridge image. Only :latest remains from a historical publish.

  2. Inconsistent architecture: authbridge-proxy (the default agent sidecar) already obtains SPIFFE credentials in-process via the go-spiffe WorkloadAPI client — "no spiffe-helper sidecar binary" (Dockerfile comment). The operator using a separate sidecar was inconsistent with this direction.

Changes

clientregistration_controller.go: replace JWTSVIDPath + os.ReadFile with workloadapi.New() + FetchJWTSVID(). Uses the same socket address as verifiedFetch (--verified-fetch-spiffe-socket). Removes path traversal validation (no longer needed).

cmd/main.go: remove --jwt-svid-path flag; SpiffeSocket field on the reconciler reuses verifiedFetchSpiffeSocket.

manager.yaml: remove spiffe-helper sidecar container, jwt-svid emptyDir volume, spiffe-helper-config volume mount. The spiffe-workload-api CSI volume remains (shared with verifiedFetch).

configmap-spiffe-helper.yaml: deleted.

values.yaml: remove jwtSVIDPath value.

Depends on

Checklist

  • All commits signed-off (DCO)
  • Backward compatible — UseSpiffeAuth=false path unchanged
  • Builds and tests pass

Assisted-By: Claude Code

Alan-Cha added 12 commits July 12, 2026 09:30
@Alan-Cha
docs(values): clarify spiffe.enabled vs spiffe.operatorAuth.enabled a…
ac91e70 
…nd jwtAudience

Add comments to values.yaml explaining the semantic distinction between the
two flags and what jwtAudience actually represents in the SPIFFE/Keycloak flow.

Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
docs(values): document jwtAudience audience mismatch pitfall
3d59b72 
Keycloak's FederatedJWTClientValidator checks the JWT-SVID aud claim against
the realm's OIDC issuer URL (always the external/public URL). Using the
internal k8s service address causes a silent audience mismatch and
invalid_client_credentials. Document the correct value and how to find it.

Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
fix(charts): remove jwtAudience value, always derive from keycloak.pu…
62f8a1b 
…blicUrl

jwtAudience was always going to be keycloak.publicUrl/realms/<realm> —
the issuer URL of the Keycloak realm the operator authenticates to.
There is no meaningful reason to configure it independently.

Remove the user-facing field and derive the audience directly in the
template. keycloak.publicUrl is already required for other reasons (UI,
authbridge) so this does not add a new dependency. Fail at helm install
time with a clear message if keycloak.publicUrl is not set.

Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
fix(charts): use v0.5.0-rc.3 as spiffe-helper default tag
d02fe80 
v0.5.0-rc.3 is the latest versioned spiffe-helper image tag confirmed to
exist in ghcr.io. Previous tag (v0.6.0-alpha.4) was a GitHub release tag
but the corresponding container image was never published. Update comment
to warn that not all release tags produce container images.

Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
fix(controller): skip credential secret creation in federated-jwt mode
aae42d3 
In federated-jwt mode, agents authenticate to Keycloak using their SPIFFE
identity (JWT-SVID) directly via the workload API socket. The operator still
registers the Keycloak client with federated-jwt authenticator type, but no
credential Secret is needed — AuthBridge reads JWT-SVIDs directly, not from
a mounted client-secret file.

Previously the operator wrote a Kubernetes Secret containing the Keycloak-
generated client secret even in federated-jwt mode, even though nothing ever
reads it. This is a hygiene issue (unnecessary secrets in agent namespaces)
and a defense-in-depth gap (secret exists even if Keycloak correctly rejects
it). Skipping creation removes the noise.

Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
fix(charts): remove noisy comment from spiffe-helper tag
22aaa58 
Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
revert(charts): remove spiffeHelper.image values, hardcode :latest in…
573b1f7 
… template

The spiffeHelper.image.* values block was introduced to pin the sidecar image,
but ghcr.io/kagenti/kagenti-extensions/spiffe-helper is not published by the
kagenti-extensions automated pipeline and the pinned tag (v0.6.0-alpha.4) does
not exist in the registry. Remove the values block entirely and hardcode :latest
in the template, which is the only published image. A separate issue will track
the correct long-term approach.

Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
fix(controller): also skip patchTemplate annotation in federated-jwt …
f07ac43 
…mode

When skipping credential secret creation in federated-jwt mode, also skip
annotating the pod template with the secret name. The webhook mounts the
credential secret based on that annotation — if we skip secret creation but
still set the annotation, pods get stuck in Init because the secret does
not exist.

In federated-jwt mode, AuthBridge reads JWT-SVIDs from the SPIFFE workload
API socket directly and does not need the credential secret at all.

Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
fix(webhook): skip credential annotation in federated-jwt mode
8cac96c 
The authbridge webhook pre-populates the Keycloak client-credentials
annotation so the credential Secret volume is ready before the controller
reconciles. In federated-jwt mode, no credential Secret is created —
AuthBridge reads JWT-SVIDs from the SPIFFE workload API socket directly.
Injecting the volume mount for a non-existent Secret left pods stuck in
Init:0/1.

Fix: read CLIENT_AUTH_TYPE from authbridge-config before pre-populating
the annotation, and skip it when federated-jwt is configured.

Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
fix(charts): restore spiffeHelper.image values block with latest as d…
77598e9 
…efault

Hardcoding :latest in the template was worse — it removed the ability to
override the image via Helm values without forking the chart. Restore the
templated approach with tag: latest as the default, which is the only
reliably-published tag.

Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
revert: back to hardcoded spiffe-helper:latest, remove values block
2e6ad42 
Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
feat(operator): fetch JWT-SVID directly via go-spiffe SDK, remove spi…
0532b06 
…ffe-helper sidecar

The spiffe-helper sidecar approach used an abandoned standalone image from
the kagenti-extensions pipeline (removed April 2026 when spiffe-helper was
bundled into authbridge). The authbridge-proxy itself already uses the
go-spiffe WorkloadAPI client in-process — this change makes the operator
consistent with that architecture.

Changes:
- clientregistration_controller.go: replace JWTSVIDPath + os.ReadFile with
  workloadapi.New() + FetchJWTSVID(). Uses the same socket as verifiedFetch.
  Removes path traversal validation (no longer needed without file I/O).
- cmd/main.go: remove --jwt-svid-path flag; reuse --verified-fetch-spiffe-socket
  for JWT-SVID fetching via SpiffeSocket field on the reconciler.
- manager.yaml: remove spiffe-helper container, jwt-svid emptyDir volume,
  spiffe-helper-config volume mount. spiffe-workload-api CSI volume remains
  (shared with verifiedFetch).
- configmap-spiffe-helper.yaml: deleted (no longer needed).
- values.yaml: remove jwtSVIDPath value.

Signed-off-by: Alan Cha <alan.cha@ibm.com>
Assisted-By: Claude (Anthropic AI) <noreply@anthropic.com>
Signed-off-by: Alan Cha <Alan.cha1@ibm.com>
@Alan-Cha
Alan-Cha requested a review from a team as a code owner July 13, 2026 14:22
@Alan-Cha
Alan-Cha marked this pull request as draft July 13, 2026 14:24
@clawgenti clawgenti mentioned this pull request Jul 20, 2026
Closed
This was referenced Aug 10, 2026
Closed
Closed
This was referenced Aug 18, 2026
Closed
Open
@Alan-Cha Alan-Cha mentioned this pull request Aug 27, 2026
Open
@Alan-Cha

Alan-Cha commented Aug 27, 2026

Copy link
Copy Markdown
Member Author

Closing in favor of clean implementation in #522 which removes spiffe-helper sidecar and uses go-spiffe SDK directly.

@Alan-Cha Alan-Cha closed this Aug 27, 2026
@github-project-automation github-project-automation Bot moved this from New/ToDo to Done in Rossoctl Issue Prioritization Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

Rossoctl Issue Prioritization
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

feat: replace spiffe-helper sidecar with direct go-spiffe SDK call for operator JWT-SVID auth

2 participants

@Alan-Cha @rubambiza