feat(sdk): add OAuth client credentials authentication across SDKs

[sjenning]

User Story

As an operator running CI jobs or long-lived service automation, I want every official OpenShell SDK to acquire, attach, and renew gateway access tokens with the OAuth 2.0 client credentials grant, so that I can use Python, TypeScript, or Go without interactive login, prerequisite CLI invocations, client reconstruction, or custom token-refresh code.

Problem Statement

Client-credentials authentication is inconsistent across the SDKs. Python can attach a callable bearer token, but its built-in active-gateway refresher only supports refresh-token grants. TypeScript accepts only a static OIDC token. Go can perform a one-shot client-credentials exchange and separately refresh arbitrary caller-provided token sources, but it does not connect those capabilities into a renewable client authentication workflow.

Because client-credentials grants normally return no refresh token, all three SDKs lack a complete first-class path that repeats the grant before expiry and keeps the main SDK client authenticated for its lifetime.

Impact / Why This Matters

Today, service automation must shell out to the CLI, manually acquire tokens and recreate clients, or independently implement discovery, exchange, caching, concurrency, renewal, and bearer attachment. The workaround differs by language and duplicates security-sensitive behavior such as issuer validation, redirect refusal, TLS requirements, audience/scope handling, and secret redaction.

This creates portability gaps between official SDKs and makes long-running jobs fail after the initial access token expires even though their service-account credentials remain valid. It also increases the chance that individual applications will leak a client secret, request the wrong claims, stampede an identity provider, or follow a redirect that replays credential-bearing request data.

Technical Context

The gateway is already grant-agnostic: it validates bearer JWT signature, issuer, audience, expiry, roles, and scopes, then applies authorization. No gateway or protobuf change is required. The missing capability belongs at each SDK's auth-provider boundary.

The CLI establishes the canonical exchange semantics: validated OIDC discovery, no redirects, client_secret_post, no implicit interactive scopes, configured scopes, optional audience, and access-token expiry. Gateway metadata already stores issuer, client ID, audience, and scopes. The existing Keycloak E2E proves that a client-credentials service-account subject can access the gateway after receiving the required authorization.

Proposed Design

Expose an idiomatic, explicit client-credentials authentication API in the Python, TypeScript, and Go SDKs and integrate it with each SDK's main client transport. Each implementation should acquire and cache an access token in memory, repeat the client-credentials grant before expiry, coalesce concurrent exchanges, and attach the current bearer without requiring client reconstruction.

All SDKs should accept explicit issuer, client ID, client secret, scopes, and optional audience. SDKs that already consume registered gateway state should also resolve issuer, client ID, audience, and scopes from metadata.json; whether TypeScript gains registered-gateway resolution or remains explicit-only is a design decision. Preserve all existing static-token APIs for backward compatibility.

Keep the client secret only in memory or behind a caller-supplied secret provider. Never write it to token caches, logs, exceptions, or object representations. Pin shared security and expiry semantics with a cross-language conformance matrix or fixtures while allowing language-idiomatic API names and types.

Acceptance Criteria

• Python, TypeScript, and Go each expose a documented public client-credentials authentication API usable by their main SDK client.
• Every SDK supports explicit issuer, client ID, client secret, scopes, and optional audience; registered-gateway construction resolves those fields wherever that SDK supports gateway metadata.
• Each SDK acquires a service-account access token and attaches it as authorization: Bearer <token> to all supported RPC call shapes.
• Long-running clients repeat the client-credentials grant before token expiry without requiring client reconstruction.
• Concurrent requests share one in-flight acquisition or renewal, and cancellation by one waiter does not corrupt shared token state.
• Client-credentials requests do not implicitly add interactive scopes such as openid, profile, or email.
• OIDC discovery validates the configured issuer, and token requests do not follow redirects that could replay the client secret.
• Remote token endpoints require protected transport by default; explicit development exceptions do not silently weaken production behavior.
• Client secrets are never persisted or exposed in errors, logs, request diagnostics, or object representations.
• Invalid credentials, missing configuration, malformed responses, missing or invalid expiry data, and transport failures produce actionable SDK errors without leaking sensitive request data.
• Existing static-token and caller-provided auth-provider APIs remain backward compatible.
• Shared conformance coverage pins request fields, audience/scope behavior, expiry/renewal, concurrency, redirect/TLS posture, and secret handling across all three SDKs.
• SDK-specific unit and integration tests exercise acquisition, automatic renewal, bearer attachment, and failure behavior.
• Published Python, TypeScript, and Go documentation describes the service-account workflow and gateway authorization prerequisites.

Affected Components

Component	Key Files	Role
Python SDK auth and transport	python/openshell/sandbox.py, python/openshell/__init__.py	Callable bearer injection, active-gateway resolution, token renewal, lifecycle, and public exports
Python tests and E2E	python/openshell/sandbox_test.py, e2e/python/oidc/helpers.py, e2e/python/oidc/oidc_auth_test.py	Existing refresh/security coverage and Keycloak client-credentials fixture
TypeScript SDK auth and transport	sdk/typescript/src/transport.ts, sdk/typescript/src/client.ts, sdk/typescript/src/index.ts	Static bearer interceptor, client factories, and public API surface
TypeScript tests and docs	sdk/typescript/src/transport.test.ts, new auth tests, sdk/typescript/README.md	Transport guards, renewable auth coverage, and usage documentation
Go SDK OIDC and auth	sdk/go/openshell/v1/oidc/credentials.go, sdk/go/openshell/v1/auth_refresh.go, sdk/go/openshell/v1/client.go	One-shot exchange, generic token-source caching, and client attachment
Go registered-gateway support	sdk/go/openshell/v1/gateway/config.go, sdk/go/openshell/v1/gateway/gateway.go, sdk/go/openshell/v1/gateway/token.go	Metadata parsing and current disk-backed OIDC provider
Go tests and docs	sdk/go/openshell/v1/oidc/credentials_test.go, sdk/go/openshell/v1/auth_refresh_test.go, sdk/go/README.md, sdk/go/docs/src/api/oidc.md	Existing exchange/renewal coverage and public documentation
Canonical metadata and CLI behavior	crates/openshell-bootstrap/src/metadata.rs, crates/openshell-bootstrap/src/oidc_token.rs, crates/openshell-cli/src/oidc_auth.rs	Shared gateway fields, token-bundle contract, and reference exchange semantics

Technical Investigation

Architecture Overview

Each SDK already has a usable transport/auth boundary but exposes a different subset of the required lifecycle:

• Python: the gRPC interceptor calls a bearer provider per RPC. The active-gateway provider caches and renews authorization-code tokens with a refresh token, but it cannot repeat a client-credentials grant.
• TypeScript: the Connect interceptor closes over a static oidcToken. Both root and sandbox clients build that transport directly, and the README instructs users to recreate the client after refreshing a token.
• Go: the OIDC package performs a secure one-shot client-credentials exchange. The core SDK accepts per-RPC credentials and has a generic, single-flight RefreshableToken, but ClientCredentials is not a reusable oauth2.TokenSource and is not directly composed into a client. The registered-gateway path rereads a token file rather than re-exchanging client credentials.

The TypeScript public API source explicitly defers an OidcRefresher until a cross-language Python/Go/TypeScript conformance suite exists, providing a natural alignment point for this issue.

Code References

Location	Description
python/openshell/sandbox.py:70	_BearerAuthInterceptor invokes the current provider and attaches bearer metadata for every gRPC call shape.
python/openshell/sandbox.py:263	SandboxClient.__init__() accepts a static token or zero-argument callable.
python/openshell/sandbox.py:323	from_active_cluster() resolves gateway metadata, TLS, and disk-backed OIDC auth.
python/openshell/sandbox.py:1198	The non-refreshing provider fails closed when a cached token expires.
python/openshell/sandbox.py:1247	_OidcRefresher caches and coordinates refresh-token renewal.
python/openshell/sandbox.py:1504	_refresh() requires refresh_token and only performs grant_type=refresh_token.
python/openshell/sandbox_test.py:1107	Existing regression test documents the client-credentials/no-refresh-token failure.
sdk/typescript/src/transport.ts:17	ConnectOptions exposes only a static oidcToken.
sdk/typescript/src/transport.ts:44	The auth interceptor closes over static token material.
sdk/typescript/src/transport.ts:70	Existing token exclusivity and plaintext-remote transport guards must be preserved.
sdk/typescript/src/client.ts:531	SandboxClient.connect() only builds the static transport.
sdk/typescript/src/client.ts:1203	OpenShellClient.connect() shares the same static transport.
sdk/typescript/src/index.ts:6	Comment explicitly reserves a cross-language, conformance-tested OIDC refresher.
sdk/typescript/README.md:47	Documentation states auth is static for the client's lifetime.
sdk/go/openshell/v1/oidc/credentials.go:21	ClientCredentials() securely performs one exchange and returns *oauth2.Token.
sdk/go/openshell/v1/auth_refresh.go:59	RefreshableToken caches arbitrary token sources with expiry, singleflight, and retry backoff.
sdk/go/openshell/v1/client.go:66	NewClient() attaches an AuthProvider as per-RPC credentials.
sdk/go/openshell/v1/gateway/gateway.go:156	Registered OIDC auth wraps a disk token source, not a client-credentials exchanger.
sdk/go/openshell/v1/gateway/token.go:90	The disk source reads cached access-token state and cannot repeat the grant.
sdk/go/openshell/v1/gateway/config.go:52	Gateway parsing exposes issuer/client ID but currently omits audience/scopes.
crates/openshell-bootstrap/src/metadata.rs:52	Canonical metadata includes issuer, client ID, audience, and scopes.
crates/openshell-bootstrap/src/oidc_token.rs:18	Client-credentials bundles intentionally contain no refresh token.
crates/openshell-cli/src/oidc_auth.rs:185	CLI flow establishes discovery, request-body secret, scopes, audience, redirect, and expiry behavior.
e2e/python/oidc/helpers.py:80	Existing E2E manually performs the exchange that SDK APIs should replace.
e2e/python/oidc/oidc_auth_test.py:174	Keycloak E2E proves service-account gateway authorization.

Current Behavior

Python

Callers can pass an already-issued token or write a custom callback. from_active_cluster() reads oidc_token.json; a CLI-created client-credentials access token works until its expiry grace window. Renewal then fails because _OidcRefresher._refresh() requires a refresh token. auto_refresh=False changes this to a fail-closed expiry error but does not reacquire a token.

TypeScript

Callers pass oidcToken to OpenShellClient.connect() or SandboxClient.connect(). The interceptor reuses that string for the client's lifetime. There is no native discovery, exchange, token-provider abstraction, renewal, or registered-gateway/XDG loader. A caller must fetch a new token and construct a new SDK client.

Go

oidc.ClientCredentials() supports explicit or gateway-derived issuer/client ID, validates discovery, sends client_secret_post, applies explicit scopes, refuses redirects, limits response size, and redacts secrets. It returns one token but does not attach or renew it. v1.RefreshableToken() can renew a caller-authored oauth2.TokenSource, but the SDK does not expose a client-credentials token source that composes the two. The gateway-aware client reads cached tokens from disk; it cannot repeat the grant and its metadata model omits audience/scopes.

The Go WithGateway documentation also says tokens are persisted, while ClientCredentials() does not persist them; the revised behavior and documentation should resolve that mismatch.

What Would Need to Change

• Python: add a public client-credentials token provider and integrate it with the existing callable/interceptor boundary, active-gateway selection, close lifecycle, and high-level Sandbox wrapper where applicable.
• TypeScript: add public client-credentials configuration and an async token provider to the Connect interceptor, with discovery, exchange, expiry cache, single-flight renewal, typed failures, and preserved edge/static-token transport guards.
• Go: expose a reusable client-credentials token source or auth-provider composition, connect it cleanly to NewClient, and extend registered-gateway metadata parsing for audience/scopes when using gateway-derived configuration.
• Cross-SDK: define shared fixtures or a conformance matrix for request fields, discovery/redirect/TLS behavior, expiry, renewal, concurrency, cancellation, and secret redaction while retaining language-idiomatic APIs.
• Docs/tests: update each SDK's public docs and add unit plus integration/E2E coverage for the complete acquire-attach-renew lifecycle.

No server, protobuf, gateway TOML, Helm, deployment, compute-driver, sandbox-infrastructure, or LSM-sensitive change is required. Python already depends on httpx; TypeScript can use Node 20's HTTP/fetch primitives or deliberately add a dependency; Go already has the required OAuth/HTTP dependencies.

Alternative Approaches Considered

1. Document each SDK's existing escape hatches. Python callbacks and Go token-source composition can be made to work, but TypeScript remains static and every application must reimplement security-sensitive behavior.
2. Acquire once in every SDK and require client reconstruction. This gives API symmetry but does not satisfy long-running automation and preserves expiry failures.
3. Teach disk-backed refreshers to infer client credentials from a missing refresh token. Absence of a refresh token does not identify the original grant, and the secret is intentionally absent from token bundles.
4. Silently detect OPENSHELL_OIDC_CLIENT_SECRET. This matches CLI convenience but makes outbound token exchanges implicit. Explicit opt-in with an optional environment-backed provider is safer.
5. Persist the client secret with access-token state. Rejected because it expands durable secret exposure and conflicts with the current bundle contract.
6. Require one byte-identical public API across languages. Rejected in favor of a shared behavioral/security contract with idiomatic language surfaces.
7. Shell out to openshell gateway login. This adds process coupling and cannot provide a robust native per-client renewal lifecycle.

Patterns to Follow

• Preserve each SDK's existing auth-provider/interceptor boundary rather than adding OAuth logic to resource methods.
• Match the CLI's client-credentials request semantics: no implicit interactive scopes, optional configured audience, and client_secret_post unless maintainers deliberately support additional token-endpoint auth methods.
• Reuse Python's expiry grace, synchronization, issuer validation, redirect refusal, and deterministic cleanup patterns.
• Reuse TypeScript's token exclusivity, cookie validation, plaintext-remote refusal, SdkError, and shared root/scoped transport.
• Reuse Go's secure one-shot exchange, response bound, typed/sentinel errors, generic singleflight refresh/backoff layer, and AuthProvider attachment.
• Keep secret material out of shared token caches. Preserve existing static-token configuration in every SDK.

Scope Assessment

• Complexity: High — cross-language public APIs, security semantics, tests, docs, and conformance behavior
• Confidence: High — all transport hooks, canonical exchange semantics, and a real Keycloak fixture already exist
• Estimated files to change: 18–25, depending on registered-gateway support and shared fixture design
• Issue type: feat

Risks & Open Questions

• What idiomatic public API should each language expose while preserving one behavioral contract?
• Must TypeScript gain registered-gateway/XDG resolution, or is explicit issuer/client configuration sufficient for parity?
• Should client secrets be accepted only as strings, or also through callbacks/providers for runtime secret rotation? Environment lookup should require explicit opt-in.
• Is client_secret_post sufficient for CLI parity, or should SDKs support client_secret_basic for providers that require it?
• Should renewed access tokens remain memory-only or optionally write back to the shared CLI cache? Client secrets must never be persisted.
• How should missing or invalid expires_in behave so a token is not treated as permanently fresh?
• Should one UNAUTHENTICATED RPC trigger a forced re-exchange in addition to proactive expiry renewal?
• Should a renewal failure fail closed or temporarily return a cached token? Go's generic refresher currently allows stale-token fallback, while Python fails closed.
• How should cancellation interact with a shared in-flight exchange, especially when one waiter cancels but others remain active?
• Should token-endpoint transport require HTTPS except explicit loopback development, independently of the gateway transport settings?
• Is shared access-token cache interoperability in scope? Go's disk source expects RFC3339 expiry, while the canonical Rust bundle uses Unix expires_at; fixing that existing mismatch could expand the issue.
• Service-account JWTs still need configured roles/scopes and workspace membership. SDK authentication cannot manufacture gateway authorization.

Disposition Readiness

• State: state:validated
• Assessment: The per-SDK limitations are directly demonstrated in source and tests. Canonical CLI behavior, the Go one-shot implementation, all three transport hooks, and the Keycloak E2E establish feasibility. Remaining questions are public API and cross-SDK consistency decisions for accepted work, not missing evidence.
• Missing evidence: None

Test Considerations

• Add language-neutral fixtures or an explicit conformance matrix covering discovery documents, exact token request fields, scopes/audience, token responses, expiry/leeway, redirects, TLS policy, malformed responses, and errors that attempt to echo the secret.
• Python: extend current bearer/refresher unit coverage and replace or supplement the manual Keycloak exchange with the public SDK API.
• TypeScript: add auth-provider tests for acquisition, per-request bearer changes, single-flight renewal, cancellation, expiry, typed failures, and preservation of edge/static-token transport guards.
• Go: compose client credentials with the auth provider in tests, cover repeated exchange and audience/gateway metadata, and resolve the stale-token and persistence-documentation semantics.
• Add SDK-specific integration coverage against the existing Keycloak/gateway fixture, or a shared harness that proves acquire, attach, authorize, expire/renew, and call again for all three languages.
• Eventual implementation verification should include mise run test:python, mise run sdk:ts:ci, mise run go:ci, mise run e2e:oidc-python:docker, mise run pre-commit, and mise run ci, plus any new TypeScript/Go OIDC E2E task introduced by the work.

Documentation Impact

Update the published gateway-auth/service-account documentation plus each SDK's own usage surface: Python SDK examples under docs/, sdk/typescript/README.md, sdk/go/README.md, and the generated Go OIDC/auth docs. architecture/gateway.md already describes client-credentials login; update it only if the SDK/cache ownership boundary becomes a stable architectural invariant.

No gateway TOML fields or defaults change, so docs/reference/gateway-config.mdx does not require an update. Helm, compute-driver, deployment, and LSM documentation are unaffected.

---

Updated by spike investigation. state:validated means the issue is ready for human disposition; state:needs-info means specific evidence is still required. A human applies state:accepted or places the issue on the roadmap if OpenShell should pursue the work. To queue unattended agent planning, a human applies agent:plan-requested; a direct request to an agent can instead invoke build-from-issue.

[sjenning]

🏗️ build-plan

Implementation Plan

Issue type: feat
Complexity: High
Confidence: High — the gateway is already grant-agnostic and every SDK has a per-RPC auth hook; the work is bounded to SDK auth, tests, and docs.

Summary

Add an idiomatic client-credentials auth provider to Python, TypeScript, and Go. Each provider will lazily discover the issuer, exchange with client_secret_post, cache only the access token in memory, renew within a shared 30-second leeway, coalesce concurrent exchanges, and fail closed when renewal is required but fails. Existing static-token and caller-provided auth APIs remain unchanged.

Resolved Contract

• Public APIs: Python exports ClientCredentialsAuth; TypeScript exports clientCredentials(options): OidcTokenProvider; Go adds oidc.NewClientCredentialsAuth(opts ...LoginOption) (types.AuthProvider, error) while preserving the existing one-shot API.
• Gateway metadata: Python and Go fill omitted issuer/client ID/audience/scopes from registered gateway metadata. TypeScript remains explicit-only because it has no registered-gateway abstraction.
• Secret shape: accept literal or language-idiomatic suppliers invoked per exchange; never include provider errors or secret values in SDK errors.
• Exchange: use client_secret_post, no implicit scopes, optional configured scope and audience.
• Storage/lifecycle: access tokens and secrets remain memory-only; renew lazily before RPCs without touching oidc_token.json.
• Expiry/failure: require finite positive expires_in; renew inside a 30-second leeway and fail closed if renewal fails.
• Concurrency: share one in-flight exchange; failed exchanges clear state for later retry. TypeScript and Go waiters cancel independently without poisoning shared work.
• Transport/security: validate issuer equality, reject URL userinfo/fragments, require HTTPS except literal loopback HTTP, and refuse redirects for discovery and token POSTs.
• Errors: preserve each SDK's typed error surface and never include form bodies, arbitrary IdP response bodies, supplier error text, or secrets.

Scope

• sdk/conformance/oauth-client-credentials.json (new): language-neutral request, scope/audience, expiry, URL, redirect, mismatch, and redaction vectors.
• python/openshell/sandbox.py, python/openshell/__init__.py, python/openshell/sandbox_test.py: implement/export ClientCredentialsAuth, wire direct and active-gateway construction, and cover acquisition/renewal/security/lifecycle.
• sdk/typescript/src/oidc.ts, sdk/typescript/src/oidc.test.ts (new): native Node client-credentials provider and conformance tests.
• sdk/typescript/src/transport.ts, transport.test.ts, index.ts: add async token-provider attachment, exclusivity checks, tests, and exports.
• sdk/go/openshell/v1/oidc/options.go, credentials.go, discovery.go and tests: reusable renewable provider, metadata defaults, secure exchange, and conformance coverage.
• sdk/go/openshell/v1/gateway/config.go, config_test.go: expose OIDC audience/scopes without changing the default disk-backed auth path.
• e2e/python/oidc/helpers.py, oidc_auth_test.py: exercise the real Keycloak/gateway path through the public Python API.
• Published SDK/auth docs and architecture/gateway.md: document all three APIs and the stable in-memory renewal boundary.

Implementation Steps

1. Add the shared contract fixture and language-specific fixture loaders.
2. Implement and test Python ClientCredentialsAuth and client integration.
3. Implement and test the TypeScript provider and async interceptor path.
4. Implement and test the Go renewable auth provider and gateway metadata fields.
5. Update the Keycloak E2E and user/SDK/architecture documentation.
6. Run focused SDK suites, OIDC E2E, pre-commit, and full CI.

Test Plan

• Unit/conformance: mise run test:python, mise run sdk:ts:ci, and mise run go:ci; every language consumes the shared fixture and covers forms, no implicit scopes, audience, issuer/URL/redirect validation, expiry, renewal, single-flight, cancellation where supported, fail-closed behavior, supplier rotation, redaction, and static-auth compatibility.
• Integration: Python httpx.MockTransport, TypeScript local Node HTTP server, and Go httptest.Server exercise discovery and repeated exchanges; transport tests verify current bearer attachment.
• E2E: extend the existing Python Keycloak lane and run mise run e2e:oidc-python:docker. The grant-agnostic gateway needs one live proof; TypeScript and Go receive SDK-specific HTTP/gRPC integration coverage.
• Final: mise run pre-commit and mise run ci.

Risks & Deliberate Deferrals

• Defer client_secret_basic, auth-method negotiation, remote plaintext overrides, background refresh, reactive RPC replay, and token-cache writeback.
• Defer TypeScript registered-gateway loading and the pre-existing cross-language oidc_token.json schema mismatch.
• Keep generic Go RefreshableToken behavior unchanged; the new provider implements the stricter fail-closed contract.
• URL validation, issuer matching, redirect refusal, bounded responses, and non-reflective errors are merge requirements because discovery and IdP responses are attacker-influenced.

Documentation Impact

• Update docs/reference/gateway-auth.mdx, docs/sandboxes/manage-sandboxes.mdx, sdk/typescript/README.md, sdk/go/README.md, Go OIDC/gateway docs, and architecture/gateway.md.
• No navigation, gateway TOML/default, Helm, compute-driver, server, protobuf, deployment, LSM, or agent-infrastructure changes are expected.

---

Revision 1 — initial plan

[sjenning]

🏗️ build-from-issue-agent

Implementation Complete

PR: #2907

What was built

Python, TypeScript, and Go now expose public renewable OAuth client-credentials providers integrated with their main SDK transports. The implementations share request, renewal, concurrency, transport, redirect, expiry, response-bound, and secret-redaction conformance coverage while preserving existing static-token APIs.

Tests

• Python SDK: 108 passed
• TypeScript SDK: 91 passed with coverage, lint, typecheck, and build checks
• Go SDK: full race, lint, build, docs, and proto CI passed
• OIDC E2E: 84 passed against Keycloak and a Docker gateway
• Repository: mise run pre-commit and full mise run ci passed

Docs updated

• Gateway authentication and sandbox management published docs
• TypeScript and Go SDK usage/API docs
• Gateway architecture lifecycle boundary

The issue will auto-close when the PR is merged.