From feefafb9c8be27b0278a733f304609697a0963df Mon Sep 17 00:00:00 2001 From: Mrunal Patel Date: Wed, 18 Nov 2026 06:24:35 +0000 Subject: [PATCH 01/10] docs(rfc): propose workspace secrets for shared refresh material Propose workspace-scoped secrets that provider refresh configuration can reference, so one shared OAuth client secret rotates for every provider. Refs #5563. Signed-off-by: Mrunal Patel --- rfc/NNNN-workspace-secrets/README.md | 669 +++++++++++++++++++++++++++ 1 file changed, 669 insertions(+) create mode 100644 rfc/NNNN-workspace-secrets/README.md diff --git a/rfc/NNNN-workspace-secrets/README.md b/rfc/NNNN-workspace-secrets/README.md new file mode 100644 index 0000000000..71a7b89908 --- /dev/null +++ b/rfc/NNNN-workspace-secrets/README.md @@ -0,0 +1,669 @@ +--- +authors: + - "@mrunalp" +state: draft +links: + - https://github.com/NVIDIA/OpenShell/issues/5563 + - https://github.com/NVIDIA/OpenShell/issues/5663 + - https://github.com/NVIDIA/OpenShell/issues/896 + - https://github.com/NVIDIA/OpenShell/pull/5139 +--- + +# RFC NNNN - Workspace Secrets + + + +## Summary + +This RFC adds **workspace secrets**: named, workspace-scoped secret values that +OpenShell stores but never injects into sandboxes. A provider's refresh +configuration can name a workspace secret in place of an inline material value. +When the gateway mints or refreshes a credential, it reads the secret's current +value. Rotating that one secret updates every provider that references it. + +Each secret declares, when it is created, the only token endpoints it may be +sent to. That set never changes, so no one can redirect a secret to a server +they control by authoring a profile. Values are write-only through the ordinary +API. A separate `RevealSecret` RPC, off by default, returns the plaintext value +to an operator or to an OIDC client that holds a dedicated scope. Every reveal +is audited. Control-plane services such as OCC use it for the OAuth +authorization-code exchange that happens before any provider exists. Sandbox +delivery and `GetProviderCredentials` never return workspace secrets. + +## Motivation + +A confidential OAuth client has one client secret shared by every user who +connects through it. OpenShell today stores refresh material per provider: each +user's provider holds its own copy of `client_secret` next to its refresh token +([#5563](https://github.com/NVIDIA/OpenShell/issues/5563)). Rotating the client +secret means updating every provider. That is not just tedious, it is not +possible without reauthorizing every user: +`ConfigureProviderRefresh` replaces the whole material set and schedules every +existing secret handle for deletion +(`crates/openshell-server/src/grpc/provider.rs`). To change `client_secret` the +caller must resend `refresh_token`. With rotating refresh tokens the gateway has +replaced that token since it was configured, and it never returns it. + +OCE's OAuth connection flow exercises exactly this. An administrator registers +the confidential client once. Each user then completes consent, and OCC +exchanges the authorization code using the client secret. The resulting tokens +become that user's OpenShell provider. OCE's conformance test expects refreshes +after a client-secret rotation to use the new secret. OCC also needs the secret +*before* any provider exists, which today forces OCE to keep its own copy +outside OpenShell. + +Related work makes this more pressing. +[#5663](https://github.com/NVIDIA/OpenShell/issues/5663) adds +`client_secret_basic` and resource indicators to refresh, so OpenShell can own +refresh for more OAuth servers. [RFC 0023](https://github.com/NVIDIA/OpenShell/pull/5139) +moves all secret values behind the credential issuer, and the owner kinds it +introduces give this feature a natural home. + +This needs an RFC because it adds a new public resource type and a new +plaintext export path with its own authorization rule. It also lets one +caller's configuration send a value that caller never supplied to an upstream +endpoint. That is a confused-deputy risk OpenShell does not have today. Finally, +it changes refresh state in a way that must survive mixed-version gateways. + +## Non-goals + +- Injecting workspace secrets into sandboxes or using them as provider + credentials. They are minting material only. Shared static credentials + already fit a single static `Provider` in the workspace. +- Sharing one secret across workspaces. #5563 defers installation-wide + secrets. They can later follow `ProviderProfile`'s pattern, where an omitted + workspace selector means platform scope. +- Performing the OAuth authorization-code exchange inside OpenShell. OCC keeps + that exchange; see Alternatives. +- Typed OAuth client registrations. A workspace secret is an opaque value; the + token endpoint authentication method and resource indicator stay in the + provider profile (#5663). +- Read-only references to external secret managers, such as an existing Vault + path. RFC 0023 tracks that separately. +- Overlapping old and new secret values during rotation. The authorization + server's own rotation support covers that. + +## Proposal + +### Threat model + +Workspace secrets protect a value that the caller who binds it may not know. +Today, everyone who configures refresh supplies their own inline material, so +sending that material to a token endpoint reveals nothing new. With shared +secrets, a workspace admin could import a workspace profile whose `token_url` +points at a server they control, bind a workspace secret, and rotate the +credential. The gateway would then POST the plaintext secret to that server. + +The design keeps two guarantees: + +1. Plaintext leaves OpenShell only through `RevealSecret`, or in a mint request + to an endpoint the secret's creator approved when they supplied the value. +2. Holding `openshell:all`, `provider:write`, `secret:write`, or workspace admin + alone does not reveal a secret's value. + +A caller who can write a secret can replace its value, which breaks dependent +providers but discloses nothing. A caller who controls an approved token +endpoint already receives the secret in ordinary operation. + +### Workspace secret resource + +A `Secret` is a workspace-scoped resource with standard `ObjectMeta`: name, +gateway-generated ID, labels, `resource_version`, and immutable workspace. It +holds one value and an immutable list of allowed endpoints. + +```protobuf +message Secret { + openshell.datamodel.v1.ObjectMeta metadata = 1; + string description = 2; + // Token endpoints this value may be sent to. Fixed at creation. + repeated string allowed_endpoints = 3; + // Last time the value was written. The value itself is never returned. + google.protobuf.Timestamp value_update_time = 4; +} + +message CreateSecretRequest { + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string name = 2; + string description = 3; + // Required and non-empty. + string value = 4 [(openshell.options.v1.secret) = true]; + // Required and non-empty. Absolute https URLs, compared exactly after + // normalization (lowercase scheme and host, default port removed). + repeated string allowed_endpoints = 5; + map labels = 6; + string request_id = 7; +} + +message UpdateSecretRequest { + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string name = 2; + // Absent leaves the value unchanged. Present must be non-empty. + optional string value = 3 [(openshell.options.v1.secret) = true]; + optional string description = 4; + // Absent leaves labels unchanged. Present replaces the full set. + optional SecretLabels labels = 5; + // Rejects stale writes when set. + optional uint64 expected_resource_version = 6; + string request_id = 7; +} + +message SecretLabels { + map labels = 1; +} + +message GetSecretRequest { + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string name = 2; +} + +message ListSecretsRequest { + // Only a named workspace selection is accepted. + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + int32 page_size = 2; + string page_token = 3; + string label_selector = 4; +} + +message DeleteSecretRequest { + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string name = 2; + bool allow_missing = 3; + string request_id = 4; +} + +message DeleteSecretResponse { + DeletionOutcome outcome = 1; +} + +message RevealSecretRequest { + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string name = 2; +} + +message RevealSecretResponse { + string value = 1 [(openshell.options.v1.secret) = true]; + google.protobuf.Timestamp value_update_time = 2; +} +``` + +Paging and label-selector semantics match `ListProvidersRequest`, except that +`all_workspaces` is rejected. Get and List return metadata only. Values are +bounded by the existing provider map value limit. `allowed_endpoints` cannot be +updated. Moving a client to a new token endpoint means creating a new secret, +and only someone who knows the value can do that. + +The gateway persists a private `StoredWorkspaceSecret` record holding the +current `CredentialHandle` and pending handle deletions. Neither appears on the +public `Secret`. Values are stored through the `CredentialIssuer` trait with a +new owner kind, `WorkspaceSecret { id }`. Its legacy driver scope is +`workspace-secret-`, following the pattern phase 1 of RFC 0023 used for SSH +host keys and launch material. + +`UpdateSecret` stages a new handle and swaps it in with a compare-and-swap. It +records the superseded handle as a pending deletion and then deletes it. A +failed swap deletes the staged handle. `DeleteSecret` tombstones the record by +setting its deletion time, deletes the current and pending handles, then +removes the row. The periodic refresh worker tick also sweeps workspace-secret +tombstones and pending deletions, so a backend failure is retried rather than +leaking a handle. The next `UpdateSecret` or `DeleteSecret` also retries them. + +The CLI adds `openshell secret create|update|get|list|delete`. Values come from +`--value-env VAR` or `--value-file PATH`, never from a command-line argument. +`--allowed-endpoint URL` is repeatable on `create`. The CLI has no reveal +command. + +### Referencing secrets from refresh configuration + +`ConfigureProviderRefreshRequest` gains a map from material key to secret name: + +```protobuf +message ConfigureProviderRefreshRequest { + // ...existing fields... + // Material supplied by a workspace secret in the same workspace. Each key is a + // secret material key; each value names a Secret. + map material_secrets = 9; +} + +message ProviderCredentialRefreshStatus { + // ...existing fields... + // Material key to referenced secret name. + map material_secrets = 14; +} +``` + +```shell +openshell provider refresh configure alice-mcp \ + --credential-key MCP_ACCESS_TOKEN \ + --strategy oauth2-refresh-token \ + --material client_id="$CLIENT_ID" \ + --secret-material-env refresh_token=REFRESH_TOKEN \ + --material-secret client_secret=acme-oauth-client +``` + +Validation rules: + +- A `material_secrets` key must be secret material: a strategy secret key, such + as `client_secret` or `aws_secret_access_key`, or a key the profile declares + `secret: true`. Configuration-time keys such as `token_url`, `scope`, + `refresh_before_seconds`, and `sts_endpoint_url` are rejected rather than + silently ignored. +- Keys the minter writes back are provider-owned and cannot be referenced. For + `oauth2_refresh_token` that is `refresh_token`. After a rotating exchange the + engine stores the replacement token under that key on the provider. A shared + binding would leave each provider holding both a handle and a binding, and the + other providers would keep reading a spent token. The motivating flow shares + `client_secret`, and each provider keeps rotating its own refresh token. +- A key may appear in `material` or `material_secrets`, not both. +- Every `material_secrets` key must also be listed in the existing + `secret_material_keys` field, and the request is rejected otherwise. The CLI + adds them automatically. This makes the request wire-safe against older + replicas. An older replica ignores field 9 but already rejects a + `secret_material_keys` entry with no inline value, with `INVALID_ARGUMENT` and + before any state change. A new replica accepts an entry backed by either + `material` or `material_secrets`. +- Required-material checks, the AWS source-key pairing rule, and #5663's "Basic + requires `client_secret`" rule all count a referenced key as present. +- The secret must exist in the provider's workspace. The effective token + endpoint must be in its `allowed_endpoints`. That endpoint is the one the + minter will call: a `token_url`, an Entra URL derived from `tenant_id`, or + the Google default. +- References are limited to the OAuth2 and Google service account strategies. + `aws_sts_assume_role` is rejected until its STS client enforces the same + endpoint and redirect rules. + +The gateway binds each key to the secret's immutable ID and records its +immutable name next to it, so status reads need no secret lookup. Public callers +use names, and stored state uses IDs, following +[proto/README.md](../../proto/README.md). Configuring a reference needs no scope +beyond what `ConfigureProviderRefresh` already requires. The endpoint +constraint, not a second scope, is what prevents disclosure. + +### Minting with the current value + +Before each mint, the refresh engine resolves inline secret handles as it does +today. It then asks a resolver port for each referenced secret's *value*. The +gateway implements that port: it reads the secret record and resolves its +current handle through the issuer. If the backend reports the handle missing and +the record's handle has changed in the meantime, the port re-reads the record +and retries once. A mint that races `UpdateSecret` therefore uses the new value +instead of failing. + +Immediately before sending the request, the minter checks the effective endpoint +against every referenced secret's `allowed_endpoints`. This mint-time check is +the authoritative one; the configure-time check only reports errors early. A +mismatch fails with `fix_configuration` and failure code +`secret_endpoint_not_allowed`, and the request is never sent. + +The check must hold for the whole HTTP exchange, not just the first URL. The +OAuth minter's HTTP client follows redirects by default, and a 307 or 308 +redirect resends the request body to the new location. Mints that carry a +workspace secret therefore disable automatic redirects. Any redirect response +fails the mint with `fix_configuration` and failure code +`secret_endpoint_redirected`, and the client never contacts the redirect target. + +```mermaid +sequenceDiagram + participant Admin as Admin / OCE + participant GW as Gateway + participant ISS as Credential issuer (embedded) + participant AS as OAuth server + Admin->>GW: UpdateSecret(acme-oauth-client, new value) + GW->>ISS: store new handle, CAS swap, delete old + Note over GW,ISS: no snapshot publish: secret is never injected + ISS->>GW: resolve referenced secret value (resolver port) + GW-->>ISS: current value, retrying once on a concurrent swap + ISS->>ISS: check effective endpoint against allowed_endpoints + ISS->>AS: refresh_token grant with current client secret + AS-->>ISS: access token +``` + +A rotation takes effect at each provider's next mint. Rotation does not trigger +a mint: existing access tokens stay valid, and a mint for every dependent +provider at once would add load at the authorization server. Callers who want to +confirm a new value can force `RotateProviderCredential` on one provider. +Workspace secrets are never delivered to supervisors, so rotation does not +publish provider snapshots or change provider environment revisions. + +Failure classification: + +| Condition | Recovery action | Failure code | +|---|---|---| +| Backend resolve fails after the retry | `retry` (short backoff) | `referenced_secret_unavailable` | +| Secret record missing or tombstoned | `fix_configuration` | `referenced_secret_missing` | +| Effective endpoint not allowed | `fix_configuration` | `secret_endpoint_not_allowed` | +| Token endpoint responds with a redirect | `fix_configuration` | `secret_endpoint_redirected` | +| Authorization server rejects the client | existing OAuth classification | existing codes | + +### Authorization and export boundary + +| RPC | Auth | Role | Scope | Audited | +|---|---|---|---|---| +| `CreateSecret`, `UpdateSecret`, `DeleteSecret` | bearer | workspace admin | `secret:write` | yes, naming the secret | +| `GetSecret`, `ListSecrets` | bearer | workspace user | `secret:read` | no | +| `RevealSecret` | bearer, plus the handler check below | workspace admin | `secret:reveal` | yes, every call | + +`RevealSecret` fails closed in its handler, whatever the gateway's +authentication mode: + +- It is disabled unless gateway configuration sets `enable_secret_reveal = + true`. Deployments without OCC never expose it. +- `Principal::Operator` (operator mTLS) is accepted. `bearer` methods already + admit operators with platform-admin rights, and the audit path already + records operator principals. Non-OIDC OCC deployments use this path. +- `Principal::User` is accepted only when OIDC is configured with scope + enforcement enabled, the token carries the literal `secret:reveal` scope, and + the caller is a workspace admin. `openshell:all` does not satisfy the check. + Unscoped OIDC gateways, user mTLS certificates, and local unauthenticated + development modes are rejected. Without this check, middleware skips scope + enforcement when `scopes_claim` is empty, and an empty `admin_role` grants + platform admin. +- Every other principal is rejected. + +The handler needs the gateway's scope-enforcement setting, which lives in the +authorization router today, so `ServerState` must expose it. The audit catalog +test currently requires the catalog to equal the set of `:write`-scoped RPCs. It +must allow this one audited read. + +Workspace secrets stay out of every runtime export path by construction, the +same way refresh material does today. Sandbox snapshots and +`GetProviderCredentials` read only `Provider.credentials` and +`Provider.credential_handles`. Workspace secrets live in their own records and +are resolved only through the refresh resolver port and `RevealSecret`. A test +asserts that a referenced secret's value never appears in a provider environment +snapshot or a `GetProviderCredentials` response. + +### Deletion and lifecycle + +`DeleteSecret` fails with `FAILED_PRECONDITION` while any live refresh state in +the workspace references the secret, and the error names the dependent +providers. This mirrors the provider-profile in-use guard. The check uses a +workspace-scoped refresh-state query and ignores tombstoned refresh states +awaiting cleanup. Delete and check run under `MutationScope::Workspace`, which +conflicts with `ConfigureProviderRefresh`, so a new reference cannot appear +between the check and the delete. Deleting a provider or its refresh +configuration removes its references. Workspace deletion is blocked while +secrets exist; the secret type joins the resource kinds that workspace deletion +checks. + +Bindings use immutable IDs. Deleting a secret and recreating one with the same +name therefore never silently rebinds a provider. + +### Converting references back to inline material + +`ConfigureProviderRefresh` cannot turn a reference back into inline material +for an active grant. It replaces the whole material set, so the caller would +have to resend a refresh token the gateway may have rotated and never returns. +A dedicated RPC performs the conversion instead: + +```protobuf +// Copy each referenced secret's current value into provider-owned refresh +// material and remove the bindings. Keeps all other material handles. +rpc InlineProviderRefreshSecrets(InlineProviderRefreshSecretsRequest) + returns (InlineProviderRefreshSecretsResponse); + +message InlineProviderRefreshSecretsRequest { + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string provider = 2; + string credential_key = 3; + string request_id = 4; +} + +message InlineProviderRefreshSecretsResponse { + ProviderCredentialRefreshStatus status = 1; +} +``` + +For each binding, the gateway resolves the secret's current value and stages it +as provider-owned refresh material under the same key. Then, in one +compare-and-swap of the refresh state, it does three things: + +- Adds the staged handles. +- Removes the bindings from the annotation. +- Records each former secret's `allowed_endpoints` in a refresh-state + annotation, so new binaries keep enforcing the endpoint constraint on the + copied value. + +Every other handle is kept, including a rotated `refresh_token`. So are the +authorization epoch, the refresh status, and the next refresh time. Placeholders +and active tokens are unaffected. If the swap fails, the staged handles are +deleted. + +The RPC requires `provider:write` and workspace admin, like +`ConfigureProviderRefresh`, and is audited. It is not gated by `enable_workspace_secrets`, +so it still works after the feature is disabled for a rollback. It never returns +the value, so it adds no disclosure path. The CLI exposes it as +`openshell provider refresh inline-secrets`. + +### Activation and mixed-version gateways + +Older replicas do not understand workspace secrets. Each write path is +protected differently: + +- **`CreateSecret` and other secret RPCs.** An older replica does not implement + them and returns `UNIMPLEMENTED`. +- **`ConfigureProviderRefresh` with `material_secrets`.** An older replica + ignores field 9. If the referenced key were optional, as `client_secret` is + for `oauth2_refresh_token`, it would configure a public client. The required + `secret_material_keys` listing prevents that. The older replica rejects the + request with `INVALID_ARGUMENT` before any state change, as described under + validation. +- **`DeleteWorkspace`.** An older replica checks a fixed list of resource kinds. + It would delete a workspace whose only resources are secrets, orphaning their + records and backend handles. No wire-level guard exists for this. The + supported guarantee is that every older replica is drained and retired before + the first secret is created. + +Workspace secrets therefore follow the sandbox-update activation pattern in +`persistence/image_activation.rs`. Each replica advertises a +`workspace_secrets_protocol` capability in its gateway membership record. New +replicas accept `CreateSecret`, and `ConfigureProviderRefresh` with non-empty +`material_secrets`, only when both of these hold: + +- The operator has set the default-off `enable_workspace_secrets` gateway setting. +- Every live membership record advertises the capability. + +Otherwise new replicas reject those requests with `FAILED_PRECONDITION`, which +names the missing condition. As for sandbox updates, the membership snapshot is +not a fence: membership can expire, and a process can connect after the check. +Enabling the setting is the operator's attestation that older replicas have been +drained and retired. Reads, updates, deletes, and reveal of existing secrets need +no gate, because only new replicas implement those RPCs. + +**Rollback condition.** Before rolling back to a binary without workspace-secret +support, the operator must: + +1. Disable `enable_workspace_secrets`. This stops new secrets and new references. +2. Run `InlineProviderRefreshSecrets` on the upgraded gateway for every + credential that refresh status shows with a referenced secret. Active + grants, including rotated refresh tokens, are preserved, and no user needs + to reauthorize. +3. Delete all secrets, which the delete guard now permits. + +The rollback documentation states this. `ListSecrets` in each workspace, plus +refresh status, shows what remains. Rolling back without these steps is +unsupported. In that case the protections below keep older replicas from +minting with missing material. They do not protect secrets themselves: an +older replica can still delete a workspace that holds secrets, as described +above. + +**Defense in depth for minting.** The refresh state stores bindings in +`metadata.annotations` under +`openshell.nvidia.com/refresh-material-secrets`, as a JSON map from material key +to secret ID and name. It does not use a new protobuf field. Older replicas +decode refresh state into a struct that drops unknown fields, and they rewrite +it with a compare-and-swap. This happens, for example, when they clear pending +deletions. A new field would be erased. Annotations are already preserved +across those rewrites, which is why the expiration-present and completed-mint +markers use them. + +The gateway also lists each referenced key in `secret_material_keys`. An older +replica sees a secret key with no stored handle, and +`validate_secret_material_references` refuses to mint. That refusal happens +before any state write, so the older replica does not park the refresh. It +retries and logs a warning on each sweep until a new replica takes the refresh +lock. Synchronous `GetProviderCredentials` or `RotateProviderCredential` calls +served by an older replica fail with `FAILED_PRECONDITION`. Neither path mints +with missing material. The new replica treats a key as satisfied by either a +stored handle or a binding. + +## Implementation plan + +The work ships incrementally behind the default-off `enable_workspace_secrets` +activation. Reveal has its own default-off setting. + +1. **Resource and storage.** Add the `Secret` proto and messages, a private + `StoredWorkspaceSecret` record and object type, the `WorkspaceSecret` owner + kind, CRUD RPCs, `secret:read` / `secret:write` scopes, mutation audit naming, + pending-deletion sweeps, the workspace-delete guard, CLI commands, and + storage-schema inventory updates. Add the membership capability and the + activation check. Test creation against a membership set that includes an + older replica. +2. **Refresh references.** Add `material_secrets` and the status field, + annotation-backed bindings, the resolver port with retry on concurrent swap, + configure-time and mint-time endpoint checks, redirect-free secret-bearing + mints, validation including the required `secret_material_keys` listing, the + in-use delete guard, the activation check on configure, and + `InlineProviderRefreshSecrets` with its CLI command and audit catalog entry. + Test the following: + - rotation between refreshes against a mock token endpoint + - a mint racing `UpdateSecret` + - a profile pointing at a disallowed endpoint + - a redirect from an approved endpoint, where the redirect target must receive + no request + - rejection of a referenced `refresh_token` + - Basic client authentication, once #5663 lands + - a new-client `ConfigureProviderRefresh` request with `material_secrets` sent + directly to the previous release binary, which must be rejected before any + state change + - a refresh sweep on the previous release binary over a refresh state with a + binding, which must not mint + - rollback: rotate a user's refresh token, disable `enable_workspace_secrets`, inline + the binding, delete the secret, and verify the provider mints on the + previous release +3. **Reveal.** Add `RevealSecret`, `enable_secret_reveal`, the `secret:reveal` + scope, the fail-closed handler check, and audited reads. Test denial for + `openshell:all`, `secret:read`, non-admin users, unscoped OIDC, user mTLS, + local development modes, and the disabled default. Test acceptance for an + operator and a scoped OIDC admin. +4. **Docs and SDKs.** Document workspace secrets, `--material-secret`, and the + activation, rollback, and reveal settings. Update + `docs/how-it-works/gateways/configuration.mdx` for `enable_workspace_secrets` and + `enable_secret_reveal`. The TypeScript SDK's curated API omits reveal; + `@nvidia/openshell-sdk/raw` exposes it for control-plane callers. Open + matching changes in the Go and Python SDK repositories. +5. **Align with RFC 0023.** When RFC 0023 phase 2 introduces versioned + references, a workspace secret becomes a `CredentialStore` reference owned by + `workspace_secret`. RFC 0023 then needs three changes. `CreateGrantRequest` + gains `secret_material_references` beside inline `secret_material`, so a + remote issuer resolves workspace secrets without calling the gateway. + `ResolvePurpose` gains `SECRET_REVEAL`. The issuer refuses `workspace_secret` + references for sandbox delivery and operator export, enforces + `allowed_endpoints` itself, and can disable reveal by policy. + +## Risks + +- **A new plaintext export path.** `RevealSecret` returns values that were + previously gateway-only. Mitigations: off by default, fail-closed principal + checks, a scope outside `openshell:all`, workspace admin, an audit record for + every call, and no CLI command. A leaked OCC credential still exposes every + client secret in its workspace. +- **Endpoint constraints are brittle.** An authorization server that moves its + token endpoint breaks every dependent provider until someone creates a new + secret and reconfigures. That is deliberate, because a mutable allowlist would + reopen the confused-deputy path. Mitigation: the failure code names the cause. + Exact matching after normalization avoids surprising prefix matches. +- **Larger blast radius on bad rotation.** One wrong value breaks refresh for + every dependent provider at its next mint. Mitigations: refresh status names + the referenced secret, a forced rotation on one provider checks a new value, + and updating the secret again repairs all providers. +- **Two migrations.** Shipping before RFC 0023 phase 2 means migrating workspace + secrets and annotation bindings to versioned references later. Mitigation: the + owner kind and ID-based binding match RFC 0023's model, so the migration + changes storage, not the public API. +- **Authorization special case.** `RevealSecret` checks the principal in its + handler instead of relying only on descriptor-driven middleware. Mitigation: + the check is one function, and tests cover every authentication mode. +- **Activation and rollback burden.** Operators must drain older replicas + before enabling the feature, and must inline references and delete secrets + before rolling back. Mitigation: activation reuses the sandbox-update pattern + operators already follow, `InlineProviderRefreshSecrets` makes rollback + possible without reauthorization, and the steps are documented. +- **Unsupported rollbacks.** Bringing an older binary back while secrets exist + fails closed only for minting. Older replicas log warnings and fail + synchronous calls for referenced providers rather than mint with missing + material. They can still delete a workspace that holds secrets, which orphans + backend handles that an operator must clean up by hand. + +## Alternatives + +### Do nothing + +Each provider keeps its own `client_secret` copy. Rotation requires reauthorizing +every user, because reconfiguring refresh needs the current refresh token. OCC +must also keep the client secret outside OpenShell for the first exchange. This +leaves #5563's rotation and custody requirements unmet. + +### Patch individual material keys + +Add an RPC that updates one material key without resending the rest. That fixes +rotation's need for the refresh token, but OCE must still enumerate and update +every provider. Partial failures leave stale copies, and OCC still needs its own +copy of the secret. Shared references solve all three. + +### Let OpenShell perform the authorization-code exchange + +A gateway RPC could take the authorization code, PKCE verifier, and redirect +URI, exchange them using the workspace secret, and create the provider +directly. Neither the client secret nor the user's refresh token would ever +leave OpenShell, and no reveal path would be needed. This is the stronger +security design. It was not chosen because #5563 keeps the exchange in OCC, and +because it moves OAuth flow handling into OpenShell, which RFC 0023 scopes out. +A later RFC can add it and retire `RevealSecret`. + +### Gate bindings with a scope instead of endpoint constraints + +Require a `secret:bind` scope, or restrict workspace secrets to platform +profiles. A scope protects only against tokens that lack it, and any workspace +admin granted it could still redirect a secret. Restricting secrets to platform +profiles blocks OCE, which authors workspace profiles. Binding the value to the +endpoints its owner approved closes the path for every caller. + +### Typed OAuth client registration + +A resource holding `client_id`, `client_secret`, and the token endpoint +authentication method, shaped like an RFC 7591 registration response. It groups +related settings, but it ties a generic primitive to OAuth. Only the secret +rotates, and the other fields fit in profiles and material. An opaque secret +also serves non-OAuth minting material, such as an AWS source key. + +## Prior art + +- **Kubernetes Secrets and `secretKeyRef`.** Consumers reference a secret by + name and read the current value at use time. Kubernetes grants value access to + anyone with `get` on the secret. This RFC separates metadata reads from value + reveal for that reason. +- **GitHub Actions organization secrets.** Secrets are shared by many + repositories and are write-only through the API, with repository access + policies set by the owner. That model is the default here; reveal is a narrow + exception for a control-plane caller. +- **OAuth 2.0 (RFC 6749 §2.3) and Dynamic Client Registration (RFC 7591).** A + confidential client's secret belongs to the client registration, and the + registration is tied to an authorization server. One secret therefore serves + many users' providers, and constraining where it is sent follows from the + registration itself. +- **RFC 0023, Credential Issuer Service.** It supplies typed secret owners, + resolution purposes, and the future versioned-reference model that workspace + secrets align with. + +## Open questions + +- Is exact normalized URL matching right for `allowed_endpoints`, or do some + authorization servers need path templates, such as per-tenant token URLs? +- Should `enable_secret_reveal` be a single gateway setting, or a + per-workspace policy that platform admins set? +- Should a platform admin be able to correct `allowed_endpoints` in place, for + example after an authorization server migration, given that platform admins + are already trusted with operator export? From 7b6168367cc7d7bdb0068b12a89ffa8e56fd9d6f Mon Sep 17 00:00:00 2001 From: Mrunal Patel Date: Wed, 18 Nov 2026 07:05:16 +0000 Subject: [PATCH 02/10] feat(server): add workspace secret resource Add a workspace-scoped Secret resource whose value is stored through the credential issuer under a new WorkspaceSecret owner kind and is never returned by Get or List. Each secret declares immutable, normalized token endpoints it may be sent to. Creation is gated behind the default-off enable_workspace_secrets setting and a workspace-secrets membership capability, reusing the sandbox-update activation check. Updates stage a new value and delete the superseded handle; deletes tombstone the record and a periodic sweep retries any backend cleanup. Workspace deletion is blocked while secrets exist. Refs #5563 Signed-off-by: Mrunal Patel --- .../tests/ensure_providers_integration.rs | 35 + .../openshell-cli/tests/mtls_integration.rs | 35 + .../tests/provider_commands_integration.rs | 35 + .../sandbox_create_lifecycle_integration.rs | 35 + .../sandbox_exec_streaming_integration.rs | 5 + .../sandbox_name_fallback_integration.rs | 35 + crates/openshell-core/src/config.rs | 6 + crates/openshell-core/src/metadata.rs | 23 +- .../src/endpoint.rs | 79 ++ crates/openshell-credential-issuer/src/lib.rs | 1 + .../openshell-credential-issuer/src/owner.rs | 19 + crates/openshell-sdk/tests/client_mock.rs | 35 + crates/openshell-server/build.rs | 8 +- .../proto/workspace_secret.proto | 23 + .../src/auth/audit/catalog.rs | 3 + crates/openshell-server/src/cli.rs | 11 + crates/openshell-server/src/config_file.rs | 3 + .../openshell-server/src/gateway_members.rs | 8 + crates/openshell-server/src/grpc/mod.rs | 63 ++ .../src/grpc/mutation_audit.rs | 21 +- .../src/grpc/mutation_replay/ordinary.rs | 61 +- crates/openshell-server/src/grpc/secret.rs | 966 ++++++++++++++++++ crates/openshell-server/src/grpc/workspace.rs | 52 + .../src/persistence/image_activation.rs | 88 +- .../openshell-server/src/provider_refresh.rs | 3 + crates/openshell-server/src/storage_proto.rs | 62 +- crates/openshell-server/tests/common/mod.rs | 35 + .../tests/supervisor_relay_integration.rs | 35 + proto/openshell.proto | 141 +++ 29 files changed, 1898 insertions(+), 28 deletions(-) create mode 100644 crates/openshell-credential-issuer/src/endpoint.rs create mode 100644 crates/openshell-server/proto/workspace_secret.proto create mode 100644 crates/openshell-server/src/grpc/secret.rs diff --git a/crates/openshell-cli/tests/ensure_providers_integration.rs b/crates/openshell-cli/tests/ensure_providers_integration.rs index be3e4f2e49..f83c382692 100644 --- a/crates/openshell-cli/tests/ensure_providers_integration.rs +++ b/crates/openshell-cli/tests/ensure_providers_integration.rs @@ -468,6 +468,41 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn create_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn get_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn list_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn update_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn delete_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn update_provider( &self, request: tonic::Request, diff --git a/crates/openshell-cli/tests/mtls_integration.rs b/crates/openshell-cli/tests/mtls_integration.rs index 1017712035..594eb37c95 100644 --- a/crates/openshell-cli/tests/mtls_integration.rs +++ b/crates/openshell-cli/tests/mtls_integration.rs @@ -383,6 +383,41 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn create_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn get_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn list_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn update_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn delete_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn update_provider( &self, _request: tonic::Request, diff --git a/crates/openshell-cli/tests/provider_commands_integration.rs b/crates/openshell-cli/tests/provider_commands_integration.rs index 89247ba718..f4afad4fd9 100644 --- a/crates/openshell-cli/tests/provider_commands_integration.rs +++ b/crates/openshell-cli/tests/provider_commands_integration.rs @@ -1417,6 +1417,41 @@ impl OpenShell for TestOpenShell { )) } + async fn create_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn get_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn list_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn update_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn delete_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + type WatchSandboxStream = tokio_stream::wrappers::ReceiverStream>; type ExecSandboxStream = diff --git a/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs b/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs index 8e7c0b190f..19e0cc09be 100644 --- a/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs +++ b/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs @@ -671,6 +671,41 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn create_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn get_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn list_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn update_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn delete_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn update_provider( &self, _request: tonic::Request, diff --git a/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs b/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs index 772a26e58e..0d7e693610 100644 --- a/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs +++ b/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs @@ -261,6 +261,11 @@ mock_gateway! { delete_provider_refresh(proto::DeleteProviderRefreshRequest) -> proto::DeleteProviderRefreshResponse; delete_provider(proto::DeleteProviderRequest) -> proto::DeleteProviderResponse; delete_provider_profile(proto::DeleteProviderProfileRequest) -> proto::DeleteProviderProfileResponse; + create_secret(proto::CreateSecretRequest) -> proto::SecretResponse; + get_secret(proto::GetSecretRequest) -> proto::SecretResponse; + list_secrets(proto::ListSecretsRequest) -> proto::ListSecretsResponse; + update_secret(proto::UpdateSecretRequest) -> proto::SecretResponse; + delete_secret(proto::DeleteSecretRequest) -> proto::DeleteSecretResponse; get_sandbox_config(proto::GetSandboxConfigRequest) -> proto::GetSandboxConfigResponse; get_gateway_config(proto::GetGatewayConfigRequest) -> proto::GetGatewayConfigResponse; update_config(proto::UpdateConfigRequest) -> proto::UpdateConfigResponse; diff --git a/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs b/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs index 497f3ce778..89c2723b81 100644 --- a/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs +++ b/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs @@ -441,6 +441,41 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn create_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn get_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn list_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn update_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn delete_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn update_provider( &self, _request: tonic::Request, diff --git a/crates/openshell-core/src/config.rs b/crates/openshell-core/src/config.rs index 6ffeaf1dce..96ae0274c2 100644 --- a/crates/openshell-core/src/config.rs +++ b/crates/openshell-core/src/config.rs @@ -309,6 +309,11 @@ pub struct Config { /// Disabled by default; not a license to run mixed-version writers. pub enable_sandbox_updates: bool, + /// Operator activation for creating workspace secrets and referencing them + /// from provider refresh configuration. Enable only after every gateway + /// that can serve traffic supports workspace secrets. Disabled by default. + pub enable_workspace_secrets: bool, + /// Disabled-by-default gateway interceptor service configs. pub gateway_interceptors: Vec, @@ -1111,6 +1116,7 @@ impl Config { auth: GatewayAuthConfig::default(), enable_websocket_tunnel: false, enable_sandbox_updates: false, + enable_workspace_secrets: false, gateway_interceptors: Vec::new(), provider_profile_sources: vec![GatewayProviderProfileSourceConfig::User], mtls_auth: MtlsAuthConfig::default(), diff --git a/crates/openshell-core/src/metadata.rs b/crates/openshell-core/src/metadata.rs index ccff87fd82..7bc4624ae3 100644 --- a/crates/openshell-core/src/metadata.rs +++ b/crates/openshell-core/src/metadata.rs @@ -6,8 +6,8 @@ //! These traits provide uniform access to `ObjectMeta` fields across all resource types. use crate::proto::{ - ObjectForTest, Provider, Sandbox, SandboxStatus, SandboxWorkloadTemplate, ServiceEndpoint, - SshSession, Workspace, WorkspaceMember, + ObjectForTest, Provider, Sandbox, SandboxStatus, SandboxWorkloadTemplate, Secret, + ServiceEndpoint, SshSession, Workspace, WorkspaceMember, }; use std::collections::HashMap; @@ -147,6 +147,25 @@ impl ObjectWorkspace for SandboxWorkloadTemplate { } } +// Implementations for Secret +impl ObjectId for Secret { + fn object_id(&self) -> &str { + self.metadata.as_ref().map_or("", |m| m.id.as_str()) + } +} + +impl ObjectName for Secret { + fn object_name(&self) -> &str { + self.metadata.as_ref().map_or("", |m| m.name.as_str()) + } +} + +impl GetResourceVersion for Secret { + fn get_resource_version(&self) -> u64 { + self.metadata.as_ref().map_or(0, |m| m.resource_version) + } +} + // Implementations for Workspace impl ObjectId for Workspace { fn object_id(&self) -> &str { diff --git a/crates/openshell-credential-issuer/src/endpoint.rs b/crates/openshell-credential-issuer/src/endpoint.rs new file mode 100644 index 0000000000..53a61e0a5d --- /dev/null +++ b/crates/openshell-credential-issuer/src/endpoint.rs @@ -0,0 +1,79 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +//! Token endpoint normalization for workspace-secret endpoint constraints. +//! +//! A workspace secret names the only token endpoints its value may be sent to. +//! Creation and every mint compare endpoints in this normalized form, so the +//! two checks cannot disagree about what a URL means. + +/// Normalize a token endpoint for exact comparison. +/// +/// Accepts absolute `https` URLs, or `http` URLs for loopback hosts (local +/// tests). Rejects user information, queries, and fragments. The result has a +/// lowercase scheme and host, no default port, and a non-empty path. +/// +/// # Errors +/// +/// Returns a short description of why the URL is not an acceptable endpoint. +pub fn normalize_token_endpoint(raw: &str) -> Result { + let url = reqwest::Url::parse(raw.trim()).map_err(|_| "must be an absolute URL")?; + let Some(host) = url.host_str().filter(|host| !host.is_empty()) else { + return Err("must include a host"); + }; + match url.scheme() { + "https" => {} + "http" if is_loopback_host(host) => {} + _ => return Err("must use https, or http for a loopback host"), + } + if !url.username().is_empty() || url.password().is_some() { + return Err("must not include user information"); + } + if url.query().is_some() || url.fragment().is_some() { + return Err("must not include a query or fragment"); + } + Ok(url.to_string()) +} + +fn is_loopback_host(host: &str) -> bool { + matches!(host, "localhost" | "127.0.0.1" | "::1" | "[::1]") +} + +#[cfg(test)] +mod tests { + use super::normalize_token_endpoint; + + #[test] + fn normalizes_case_default_port_and_empty_path() { + assert_eq!( + normalize_token_endpoint("HTTPS://Login.Example.COM:443").unwrap(), + "https://login.example.com/" + ); + assert_eq!( + normalize_token_endpoint(" https://login.example.com:8443/oauth/token ").unwrap(), + "https://login.example.com:8443/oauth/token" + ); + } + + #[test] + fn allows_http_only_for_loopback() { + assert!(normalize_token_endpoint("http://127.0.0.1:9000/token").is_ok()); + assert!(normalize_token_endpoint("http://localhost/token").is_ok()); + assert!(normalize_token_endpoint("http://[::1]:9000/token").is_ok()); + assert!(normalize_token_endpoint("http://login.example.com/token").is_err()); + } + + #[test] + fn rejects_ambiguous_or_credential_bearing_urls() { + for raw in [ + "", + "/oauth/token", + "ftp://login.example.com/token", + "https://user:pass@login.example.com/token", + "https://login.example.com/token?tenant=a", + "https://login.example.com/token#frag", + ] { + assert!(normalize_token_endpoint(raw).is_err(), "{raw}"); + } + } +} diff --git a/crates/openshell-credential-issuer/src/lib.rs b/crates/openshell-credential-issuer/src/lib.rs index 4d6a089b13..65ef5f9644 100644 --- a/crates/openshell-credential-issuer/src/lib.rs +++ b/crates/openshell-credential-issuer/src/lib.rs @@ -5,6 +5,7 @@ //! in the gateway; existing driver contracts and durable identities are unchanged. mod embedded; +pub mod endpoint; mod owner; pub mod refresh; mod refresh_material; diff --git a/crates/openshell-credential-issuer/src/owner.rs b/crates/openshell-credential-issuer/src/owner.rs index fcbf320433..259d105f7a 100644 --- a/crates/openshell-credential-issuer/src/owner.rs +++ b/crates/openshell-credential-issuer/src/owner.rs @@ -24,6 +24,10 @@ pub enum SecretOwnerKind<'a> { SupervisorInstance { sandbox_id: &'a str, }, + /// A workspace secret shared as minting material by many providers. + WorkspaceSecret { + id: &'a str, + }, } pub struct LegacyScope<'a> { @@ -54,6 +58,13 @@ impl<'a> SecretOwner<'a> { } } + pub fn workspace_secret(workspace: &'a str, id: &'a str) -> Self { + Self { + workspace, + kind: SecretOwnerKind::WorkspaceSecret { id }, + } + } + pub(crate) fn legacy_scope(self) -> LegacyScope<'a> { let (provider, id) = match self.kind { SecretOwnerKind::Provider { id, name } => (Cow::Borrowed(name), id), @@ -62,6 +73,9 @@ impl<'a> SecretOwner<'a> { Cow::Owned(format!("sandbox-instance-{sandbox_id}")), sandbox_id, ), + SecretOwnerKind::WorkspaceSecret { id } => { + (Cow::Owned(format!("workspace-secret-{id}")), id) + } }; LegacyScope { provider, @@ -93,6 +107,11 @@ mod tests { "sandbox-instance-sandbox-id", "sandbox-id", ), + ( + SecretOwner::workspace_secret("workspace-a", "secret-id"), + "workspace-secret-secret-id", + "secret-id", + ), ] { let scope = owner.legacy_scope(); assert_eq!(scope.workspace, "workspace-a"); diff --git a/crates/openshell-sdk/tests/client_mock.rs b/crates/openshell-sdk/tests/client_mock.rs index f98a180eea..2fa3f30b22 100644 --- a/crates/openshell-sdk/tests/client_mock.rs +++ b/crates/openshell-sdk/tests/client_mock.rs @@ -857,6 +857,41 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("unused")) } + async fn create_secret( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn get_secret( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn list_secrets( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn update_secret( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn delete_secret( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + async fn get_sandbox_config( &self, _: tonic::Request, diff --git a/crates/openshell-server/build.rs b/crates/openshell-server/build.rs index 6230f8d259..2fb7378e0b 100644 --- a/crates/openshell-server/build.rs +++ b/crates/openshell-server/build.rs @@ -31,9 +31,11 @@ fn main() -> Result<(), Box> { let public_proto_dir = manifest_dir.join("../../proto"); let storage_proto = storage_proto_dir.join("storage.proto"); let image_proto = storage_proto_dir.join("sandbox_image.proto"); + let secret_proto = storage_proto_dir.join("workspace_secret.proto"); println!("cargo:rerun-if-changed={}", storage_proto.display()); println!("cargo:rerun-if-changed={}", image_proto.display()); + println!("cargo:rerun-if-changed={}", secret_proto.display()); for imported_proto in [ "datamodel.proto", "openshell.proto", @@ -73,7 +75,11 @@ fn main() -> Result<(), Box> { "::openshell_core::proto::sandbox::v1", ) .file_descriptor_set_path(&descriptor_path) - .compile_with_config(proto_config, &[storage_proto, image_proto], &include_dirs)?; + .compile_with_config( + proto_config, + &[storage_proto, image_proto, secret_proto], + &include_dirs, + )?; Ok(()) } diff --git a/crates/openshell-server/proto/workspace_secret.proto b/crates/openshell-server/proto/workspace_secret.proto new file mode 100644 index 0000000000..58709c630e --- /dev/null +++ b/crates/openshell-server/proto/workspace_secret.proto @@ -0,0 +1,23 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 +syntax = "proto3"; + +// A new private format; frozen openshell.storage.v1 remains unchanged. +package openshell.storage.secret.v1; + +import "datamodel.proto"; +import "google/protobuf/timestamp.proto"; + +// Stored workspace secret. The value lives in credential storage; only its +// handle is kept here, and the handle never appears on the public Secret. +message StoredWorkspaceSecret { + openshell.datamodel.v1.ObjectMeta metadata = 1; + string description = 2; + // Normalized token endpoints the value may be sent to. Immutable. + repeated string allowed_endpoints = 3; + google.protobuf.Timestamp value_update_time = 4; + // Credential storage handle for the current value. Absent once tombstoned. + openshell.datamodel.v1.CredentialHandle value_handle = 5; + // Superseded handles whose backend deletion has not completed yet. + repeated openshell.datamodel.v1.CredentialHandle pending_handle_deletions = 6; +} diff --git a/crates/openshell-server/src/auth/audit/catalog.rs b/crates/openshell-server/src/auth/audit/catalog.rs index 3c4df0c8ec..bcea401fcb 100644 --- a/crates/openshell-server/src/auth/audit/catalog.rs +++ b/crates/openshell-server/src/auth/audit/catalog.rs @@ -46,6 +46,9 @@ const CATALOG: &[(&str, ActivityId)] = &[ (openshell!("ConfigureProviderRefresh"), UPDATE), (openshell!("DeleteProviderRefresh"), DELETE), (openshell!("RotateProviderCredential"), UPDATE), + (openshell!("CreateSecret"), CREATE), + (openshell!("UpdateSecret"), UPDATE), + (openshell!("DeleteSecret"), DELETE), (openshell!("UpdateConfig"), UPDATE), (openshell!("ApproveDraftChunk"), UPDATE), (openshell!("ApproveAllDraftChunks"), UPDATE), diff --git a/crates/openshell-server/src/cli.rs b/crates/openshell-server/src/cli.rs index b1e015fa72..f3f98c5ada 100644 --- a/crates/openshell-server/src/cli.rs +++ b/crates/openshell-server/src/cli.rs @@ -351,6 +351,11 @@ struct RunArgs { /// and settling its outstanding driver operations. See HA upgrade guidance. #[arg(long, env = "OPENSHELL_ENABLE_SANDBOX_UPDATES", default_value_t = false, action = ArgAction::Set)] enable_sandbox_updates: bool, + + /// Activate workspace secrets only after every gateway that can serve traffic supports + /// them. See the workspace secrets activation guidance. + #[arg(long, env = "OPENSHELL_ENABLE_WORKSPACE_SECRETS", default_value_t = false, action = ArgAction::Set)] + enable_workspace_secrets: bool, } pub fn command() -> Command { @@ -659,6 +664,7 @@ fn prepare_server_config_with_drivers( .map_err(|error| miette::miette!("invalid service routing configuration: {error}"))?; config.enable_sandbox_updates = args.enable_sandbox_updates; + config.enable_workspace_secrets = args.enable_workspace_secrets; config.supervisor_app_listener = file .as_ref() @@ -1613,6 +1619,11 @@ fn merge_file_into_args(args: &mut RunArgs, file: &GatewayFileSection, matches: { args.enable_sandbox_updates = enabled; } + if let Some(enabled) = file.enable_workspace_secrets + && arg_defaulted(matches, "enable_workspace_secrets") + { + args.enable_workspace_secrets = enabled; + } if let Some(mtls_auth) = &file.mtls_auth && arg_defaulted(matches, "enable_mtls_auth") { diff --git a/crates/openshell-server/src/config_file.rs b/crates/openshell-server/src/config_file.rs index b541762081..576659d098 100644 --- a/crates/openshell-server/src/config_file.rs +++ b/crates/openshell-server/src/config_file.rs @@ -171,6 +171,9 @@ pub struct GatewayFileSection { /// Explicit activation only after the compatible deployment rollout ends. #[serde(default)] pub enable_sandbox_updates: Option, + /// Explicit activation only after every serving gateway supports workspace secrets. + #[serde(default)] + pub enable_workspace_secrets: Option, #[serde(default)] pub grpc_rate_limit_requests: Option, #[serde(default)] diff --git a/crates/openshell-server/src/gateway_members.rs b/crates/openshell-server/src/gateway_members.rs index b70bf5f82f..ac5b67153b 100644 --- a/crates/openshell-server/src/gateway_members.rs +++ b/crates/openshell-server/src/gateway_members.rs @@ -19,6 +19,10 @@ use std::time::Duration; /// Initial stopped-sandbox configuration update protocol advertised by gateways. pub const SANDBOX_UPDATE_PROTOCOL: u32 = 1; +/// Initial workspace secrets protocol advertised by gateways. A gateway that +/// advertises it validates secret references and guards workspace deletion. +pub const WORKSPACE_SECRETS_PROTOCOL: u32 = 1; + const MEMBER_OBJECT_TYPE: &str = "gateway_member"; /// How long a membership record stays valid without a refresh. @@ -49,6 +53,9 @@ struct MemberPayload { /// Binary capability, not feature activation. Missing denotes old replicas. #[serde(default)] sandbox_update_protocol: u32, + /// Binary capability, not feature activation. Missing denotes old replicas. + #[serde(default)] + workspace_secrets_protocol: u32, } /// A gateway replica that was alive as of its last refresh. @@ -78,6 +85,7 @@ impl GatewayMemberIndex { replica_id: replica_id.to_string(), peer_endpoint: peer_endpoint.to_string(), sandbox_update_protocol: SANDBOX_UPDATE_PROTOCOL, + workspace_secrets_protocol: WORKSPACE_SECRETS_PROTOCOL, }; let bytes = serde_json::to_vec(&payload) .map_err(|err| PersistenceError::Encode(err.to_string()))?; diff --git a/crates/openshell-server/src/grpc/mod.rs b/crates/openshell-server/src/grpc/mod.rs index 3cfae86954..6bd29dfebb 100644 --- a/crates/openshell-server/src/grpc/mod.rs +++ b/crates/openshell-server/src/grpc/mod.rs @@ -11,6 +11,7 @@ pub mod provider; mod provider_credentials; pub mod provider_readiness; mod sandbox; +pub mod secret; pub mod supervisor_host; pub use sandbox::mint_persisted_authentication; mod service; @@ -72,6 +73,10 @@ use openshell_core::proto::{ UpdateProviderProfilesRequest, UpdateProviderProfilesResponse, UpdateProviderRequest, WatchSandboxRequest, open_shell_server::OpenShell, }; +use openshell_core::proto::{ + CreateSecretRequest, DeleteSecretRequest, DeleteSecretResponse, GetSecretRequest, + ListSecretsRequest, ListSecretsResponse, SecretResponse, UpdateSecretRequest, +}; use serde::{Deserialize, Serialize}; use std::collections::BTreeMap; use std::pin::Pin; @@ -823,6 +828,64 @@ impl OpenShell for OpenShellService { mutation_audit::audited(targets, mutation_replay::run(&self.state, request)).await } + // --- Workspace secrets --- + + async fn create_secret( + &self, + request: Request, + ) -> Result, Status> { + let targets = { + let request = request.get_ref(); + mutation_audit::scoped( + request.workspace_scope.as_ref(), + [mutation_audit::requested("secret", &request.name)], + ) + }; + mutation_audit::audited(targets, mutation_replay::run(&self.state, request)).await + } + + async fn get_secret( + &self, + request: Request, + ) -> Result, Status> { + secret::handle_get_secret(&self.state, request).await + } + + async fn list_secrets( + &self, + request: Request, + ) -> Result, Status> { + secret::handle_list_secrets(&self.state, request).await + } + + async fn update_secret( + &self, + request: Request, + ) -> Result, Status> { + let targets = { + let request = request.get_ref(); + mutation_audit::scoped( + request.workspace_scope.as_ref(), + [mutation_audit::requested("secret", &request.name)], + ) + }; + mutation_audit::audited(targets, mutation_replay::run(&self.state, request)).await + } + + async fn delete_secret( + &self, + request: Request, + ) -> Result, Status> { + let targets = { + let request = request.get_ref(); + mutation_audit::scoped( + request.workspace_scope.as_ref(), + [mutation_audit::requested("secret", &request.name)], + ) + }; + mutation_audit::audited(targets, mutation_replay::run(&self.state, request)).await + } + // --- Config / Policy --- async fn get_sandbox_config( diff --git a/crates/openshell-server/src/grpc/mutation_audit.rs b/crates/openshell-server/src/grpc/mutation_audit.rs index 1bf828e035..b9626b417b 100644 --- a/crates/openshell-server/src/grpc/mutation_audit.rs +++ b/crates/openshell-server/src/grpc/mutation_audit.rs @@ -11,10 +11,11 @@ use openshell_core::proto::{ AddWorkspaceMemberResponse, AttachSandboxProviderResponse, BeginRootfsTarStagingResponse, CreateWorkspaceResponse, DeleteProviderProfileResponse, DeleteProviderRefreshResponse, DeleteProviderResponse, DeleteSandboxResponse, DeleteSandboxTemplateResponse, - DeleteServiceResponse, DeleteWorkspaceResponse, DeletionOutcome, DetachSandboxProviderResponse, - ImportProviderProfilesResponse, ProviderProfile, RemoveWorkspaceMemberResponse, Sandbox, - SandboxResponse, SandboxTemplateResponse, ServiceEndpointResponse, - UpdateProviderProfilesResponse, WorkspaceSelector, workspace_selector::Selection, + DeleteSecretResponse, DeleteServiceResponse, DeleteWorkspaceResponse, DeletionOutcome, + DetachSandboxProviderResponse, ImportProviderProfilesResponse, ProviderProfile, + RemoveWorkspaceMemberResponse, Sandbox, SandboxResponse, SandboxTemplateResponse, + SecretResponse, ServiceEndpointResponse, UpdateProviderProfilesResponse, WorkspaceSelector, + workspace_selector::Selection, }; use openshell_ocsf::ResourceDetails; use tonic::{Code, Response, Status}; @@ -63,6 +64,7 @@ macro_rules! deletion { deletion!( DeleteSandboxTemplateResponse, + DeleteSecretResponse, DeleteServiceResponse, DeleteProviderResponse, DeleteProviderProfileResponse, @@ -128,6 +130,17 @@ impl Audited for SandboxTemplateResponse { } } +impl Audited for SecretResponse { + fn produced(&self) -> Vec> { + stored( + "secret", + self.secret + .as_ref() + .and_then(|secret| secret.metadata.as_ref()), + ) + } +} + impl Audited for ServiceEndpointResponse { fn produced(&self) -> Vec> { self.endpoint diff --git a/crates/openshell-server/src/grpc/mutation_replay/ordinary.rs b/crates/openshell-server/src/grpc/mutation_replay/ordinary.rs index a88c6abb9d..bbdb7feb88 100644 --- a/crates/openshell-server/src/grpc/mutation_replay/ordinary.rs +++ b/crates/openshell-server/src/grpc/mutation_replay/ordinary.rs @@ -28,6 +28,10 @@ use openshell_core::proto::{ UpdateProviderProfilesRequest, UpdateProviderProfilesResponse, UpdateProviderRequest, WorkspaceSelector, }; +use openshell_core::proto::{ + CreateSecretRequest, DeleteSecretRequest, DeleteSecretResponse, SecretResponse, + UpdateSecretRequest, +}; use openshell_core::{GetResourceVersion, ObjectId}; use prost::Message; use serde::{Deserialize, Serialize}; @@ -41,11 +45,11 @@ use crate::auth::principal::Principal; use crate::auth::workspace_authz::{ MinWorkspaceRole, authorize_workspace, selected_workspace_name, }; -use crate::grpc::{policy, provider, sandbox, service}; +use crate::grpc::{policy, provider, sandbox, secret, service}; use crate::persistence::{ObjectType, SetResourceVersion, Store}; use crate::storage_proto::{ StoredProviderCredentialRefreshStateV2 as StoredProviderCredentialRefreshState, - StoredProviderProfile, + StoredProviderProfile, StoredWorkspaceSecret, }; use crate::{ServerState, config_update_operation}; @@ -171,6 +175,9 @@ pub(in crate::grpc) enum Outcome { mutation_id: String, target_receipts: Vec, }, + Secret { + reference: Reference, + }, Service { reference: Reference, sandbox_id: String, @@ -707,6 +714,56 @@ ordinary_deletion!( .await } ); +fn secret_receipt(response: &Response) -> Result { + let secret = response.get_ref().secret.as_ref().ok_or_else(uncertain)?; + Ok(Outcome::Secret { + reference: Reference::new(secret), + }) +} + +async fn restore_secret(store: &Store, outcome: Outcome) -> Result { + let Outcome::Secret { reference } = outcome else { + return Err(replay_unavailable()); + }; + let stored: StoredWorkspaceSecret = reference.restore(store).await?; + Ok(SecretResponse { + secret: Some(secret::public_secret(&stored)), + }) +} + +scoped_mutation!( + CreateSecretRequest, + SecretResponse, + "CreateSecret", + secret::handle_create_secret, + Admin, + secret_receipt, + restore_secret +); +scoped_mutation!( + UpdateSecretRequest, + SecretResponse, + "UpdateSecret", + secret::handle_update_secret, + Admin, + secret_receipt, + restore_secret +); +ordinary_deletion!( + DeleteSecretRequest, + DeleteSecretResponse, + "DeleteSecret", + secret::handle_delete_secret, + async |req: &DeleteSecretRequest, state: &ServerState, principal: &Principal| { + selected_scope( + state, + principal, + req.workspace_scope.as_ref(), + MinWorkspaceRole::Admin, + ) + .await + } +); ordinary_deletion!( DeleteProviderRequest, DeleteProviderResponse, diff --git a/crates/openshell-server/src/grpc/secret.rs b/crates/openshell-server/src/grpc/secret.rs new file mode 100644 index 0000000000..3418e66b1d --- /dev/null +++ b/crates/openshell-server/src/grpc/secret.rs @@ -0,0 +1,966 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +//! Workspace secrets: shared, never-injected credential minting material. +//! +//! The gateway record holds only non-secret metadata and an opaque credential +//! storage handle. Values are written through the credential issuer under the +//! `WorkspaceSecret` owner kind and are never returned by these handlers. + +use std::collections::{BTreeSet, HashMap}; +use std::sync::Arc; + +use openshell_core::proto::datamodel::v1::{CredentialHandle, ObjectMeta}; +use openshell_core::proto::{ + CreateSecretRequest, DeleteSecretRequest, DeleteSecretResponse, DeletionOutcome, + GetSecretRequest, ListSecretsRequest, ListSecretsResponse, Secret, SecretResponse, + UpdateSecretRequest, +}; +use openshell_core::{GetResourceVersion, ObjectId, ObjectLabels, ObjectName, ObjectWorkspace}; +use openshell_credential_issuer::endpoint::normalize_token_endpoint; +use openshell_credential_issuer::{Handles, SecretOwner}; +use prost::Message; +use tonic::{Request, Response, Status}; +use tracing::warn; + +use super::validation::validate_dns1123_label; +use super::{MAX_MAP_VALUE_LEN, MAX_NAME_LEN}; +use crate::ServerState; +use crate::auth::workspace_authz::{ + MinWorkspaceRole, authorize_workspace, selected_workspace_name, +}; +use crate::pagination::Pagination; +use crate::persistence::{ + ObjectListQuery, ObjectType, PersistenceError, Store, WriteCondition, current_time_ms, +}; +use crate::storage_proto::StoredWorkspaceSecret; + +/// Logical key of the single value each secret stores. +pub const SECRET_VALUE_KEY: &str = "value"; +const MAX_SECRETS_PER_WORKSPACE: u32 = 500; +const MAX_ALLOWED_ENDPOINTS: usize = 16; +const MAX_ENDPOINT_LEN: usize = 2048; +const MAX_DESCRIPTION_LEN: usize = 1024; +const SWEEP_LIMIT: u32 = 1000; + +impl ObjectType for StoredWorkspaceSecret { + fn object_type() -> &'static str { + "workspace_secret" + } +} + +/// Project a stored record to its public form. Never carries handles. +pub fn public_secret(stored: &StoredWorkspaceSecret) -> Secret { + Secret { + metadata: stored.metadata.clone(), + description: stored.description.clone(), + allowed_endpoints: stored.allowed_endpoints.clone(), + value_update_time: stored.value_update_time, + } +} + +fn is_tombstoned(stored: &StoredWorkspaceSecret) -> bool { + stored + .metadata + .as_ref() + .is_some_and(|metadata| metadata.deletion_time.is_some()) +} + +fn owner(stored: &StoredWorkspaceSecret) -> SecretOwner<'_> { + SecretOwner::workspace_secret(stored.object_workspace(), stored.object_id()) +} + +fn value_handles(handle: CredentialHandle) -> Handles { + HashMap::from([(SECRET_VALUE_KEY.to_string(), handle)]) +} + +fn labels_json(stored: &StoredWorkspaceSecret) -> Result, Status> { + match stored.object_labels() { + Some(labels) if !labels.is_empty() => serde_json::to_string(&labels) + .map(Some) + .map_err(|e| Status::internal(format!("failed to serialize labels: {e}"))), + _ => Ok(None), + } +} + +/// Reject secret creation and new references unless the operator activated +/// workspace secrets and no observable live gateway lacks support for them. +pub async fn ensure_workspace_secrets_active(state: &ServerState) -> Result<(), Status> { + if !state.config.enable_workspace_secrets { + return Err(Status::failed_precondition( + "workspace secrets are not enabled; set enable_workspace_secrets after every gateway supports them", + )); + } + state + .store + .ensure_workspace_secrets_deployment_compatible() + .await + .map_err(|err| Status::failed_precondition(err.to_string())) +} + +fn ensure_credential_storage(state: &ServerState) -> Result<(), Status> { + if state.credentials.stores_provider_credentials() { + Ok(()) + } else { + Err(Status::failed_precondition( + "workspace secrets require configured credential storage", + )) + } +} + +fn validate_name(name: &str) -> Result<(), Status> { + if name.is_empty() { + return Err(Status::invalid_argument("name is required")); + } + if name.len() > MAX_NAME_LEN { + return Err(Status::invalid_argument(format!( + "name exceeds maximum length ({} > {MAX_NAME_LEN})", + name.len() + ))); + } + validate_dns1123_label(name, "name") +} + +fn validate_value(value: &str) -> Result<(), Status> { + if value.is_empty() { + return Err(Status::invalid_argument("value must not be empty")); + } + if value.len() > MAX_MAP_VALUE_LEN { + return Err(Status::invalid_argument(format!( + "value exceeds maximum length ({} > {MAX_MAP_VALUE_LEN})", + value.len() + ))); + } + Ok(()) +} + +fn validate_description(description: &str) -> Result<(), Status> { + if description.len() > MAX_DESCRIPTION_LEN { + return Err(Status::invalid_argument(format!( + "description exceeds maximum length ({} > {MAX_DESCRIPTION_LEN})", + description.len() + ))); + } + Ok(()) +} + +fn normalize_allowed_endpoints(endpoints: &[String]) -> Result, Status> { + if endpoints.is_empty() { + return Err(Status::invalid_argument( + "allowed_endpoints must name at least one token endpoint", + )); + } + if endpoints.len() > MAX_ALLOWED_ENDPOINTS { + return Err(Status::invalid_argument(format!( + "allowed_endpoints exceeds maximum entries ({} > {MAX_ALLOWED_ENDPOINTS})", + endpoints.len() + ))); + } + let mut normalized = BTreeSet::new(); + for endpoint in endpoints { + if endpoint.len() > MAX_ENDPOINT_LEN { + return Err(Status::invalid_argument(format!( + "allowed_endpoints entry exceeds maximum length ({} > {MAX_ENDPOINT_LEN})", + endpoint.len() + ))); + } + let value = normalize_token_endpoint(endpoint).map_err(|reason| { + Status::invalid_argument(format!("allowed_endpoints entry '{endpoint}' {reason}")) + })?; + normalized.insert(value); + } + Ok(normalized.into_iter().collect()) +} + +/// Validate labels by reusing the shared metadata validator. +fn validate_labels(labels: &HashMap) -> Result<(), Status> { + super::validation::validate_object_metadata( + Some(&ObjectMeta { + id: "validation".to_string(), + name: "validation".to_string(), + labels: labels.clone(), + ..ObjectMeta::default() + }), + "secret", + ) +} + +async fn authorized_workspace( + state: &ServerState, + request_principal: &crate::auth::principal::Principal, + workspace_scope: Option<&openshell_core::proto::WorkspaceSelector>, + role: MinWorkspaceRole, +) -> Result { + let authz = authorize_workspace( + &state.store, + &state.admin_role, + request_principal, + selected_workspace_name(workspace_scope)?, + role, + ) + .await?; + super::workspace::resolve_workspace(state.store.as_ref(), &authz.workspace).await +} + +/// Fetch a live (not tombstoned) secret by name. +pub async fn get_live_secret( + store: &Store, + workspace: &str, + name: &str, +) -> Result, Status> { + let stored = store + .get_message_by_name::(workspace, name) + .await + .map_err(|e| Status::internal(format!("fetch secret failed: {e}")))?; + Ok(stored.filter(|stored| !is_tombstoned(stored))) +} + +async fn stage_value( + state: &ServerState, + workspace: &str, + secret_id: &str, + value: &str, +) -> Result { + let values = HashMap::from([(SECRET_VALUE_KEY.to_string(), value.to_string())]); + let staging_id = format!("{secret_id}-value-{}", uuid::Uuid::new_v4()); + let mut handles = state + .credentials + .store_secrets_with_object_id( + SecretOwner::workspace_secret(workspace, secret_id), + &staging_id, + &values, + &HashMap::new(), + ) + .await?; + handles + .remove(SECRET_VALUE_KEY) + .ok_or_else(|| Status::internal("credential storage returned no secret handle")) +} + +async fn discard_staged_value( + state: &ServerState, + workspace: &str, + secret_id: &str, + handle: CredentialHandle, +) { + if let Err(err) = state + .credentials + .delete_secrets( + SecretOwner::workspace_secret(workspace, secret_id), + &value_handles(handle), + ) + .await + { + warn!(secret_id, error = %err, "failed to discard staged workspace secret value"); + } +} + +pub(super) async fn handle_create_secret( + state: &Arc, + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let req = request.into_inner(); + let workspace = authorized_workspace( + state, + &principal, + req.workspace_scope.as_ref(), + MinWorkspaceRole::Admin, + ) + .await? + .ensure_active()?; + validate_name(&req.name)?; + validate_value(&req.value)?; + validate_description(&req.description)?; + validate_labels(&req.labels)?; + let allowed_endpoints = normalize_allowed_endpoints(&req.allowed_endpoints)?; + ensure_workspace_secrets_active(state).await?; + ensure_credential_storage(state)?; + + let id = uuid::Uuid::new_v4().to_string(); + let now = openshell_core::time::timestamp_from_millis(current_time_ms()).ok(); + let handle = stage_value(state, &workspace, &id, &req.value).await?; + let mut stored = StoredWorkspaceSecret { + metadata: Some(ObjectMeta { + id: id.clone(), + name: req.name.clone(), + created_time: now, + labels: req.labels, + resource_version: 0, + annotations: HashMap::new(), + workspace: workspace.clone(), + deletion_time: None, + }), + description: req.description, + allowed_endpoints, + value_update_time: now, + value_handle: Some(handle.clone()), + pending_handle_deletions: Vec::new(), + }; + let labels = labels_json(&stored)?; + let write = state + .store + .create_if_workspace_count_below( + StoredWorkspaceSecret::object_type(), + stored.object_id(), + stored.object_name(), + &workspace, + &stored.encode_to_vec(), + labels.as_deref(), + u64::from(MAX_SECRETS_PER_WORKSPACE), + ) + .await; + let write = match write { + Ok(Some(write)) => write, + Ok(None) => { + discard_staged_value(state, &workspace, &id, handle).await; + return Err(Status::resource_exhausted(format!( + "workspace has reached the maximum of {MAX_SECRETS_PER_WORKSPACE} secrets" + ))); + } + Err(PersistenceError::UniqueViolation { .. }) => { + discard_staged_value(state, &workspace, &id, handle).await; + return Err(Status::already_exists("secret already exists")); + } + Err(err) => { + discard_staged_value(state, &workspace, &id, handle).await; + return Err(Status::internal(format!("persist secret failed: {err}"))); + } + }; + if let Some(metadata) = stored.metadata.as_mut() { + metadata.resource_version = write.resource_version; + } + Ok(Response::new(SecretResponse { + secret: Some(public_secret(&stored)), + })) +} + +pub(super) async fn handle_get_secret( + state: &Arc, + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let req = request.into_inner(); + if req.name.is_empty() { + return Err(Status::invalid_argument("name is required")); + } + let workspace = authorized_workspace( + state, + &principal, + req.workspace_scope.as_ref(), + MinWorkspaceRole::User, + ) + .await? + .name; + let stored = get_live_secret(&state.store, &workspace, &req.name) + .await? + .ok_or_else(|| Status::not_found("secret not found"))?; + Ok(Response::new(SecretResponse { + secret: Some(public_secret(&stored)), + })) +} + +pub(super) async fn handle_list_secrets( + state: &Arc, + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let req = request.into_inner(); + let workspace = authorized_workspace( + state, + &principal, + req.workspace_scope.as_ref(), + MinWorkspaceRole::User, + ) + .await? + .name; + if !req.label_selector.is_empty() { + super::validation::validate_label_selector(&req.label_selector)?; + } + let pagination = Pagination::new( + req.page_size, + &req.page_token, + "ListSecrets", + &[&workspace, &req.label_selector], + )?; + let after = pagination.object_cursor()?; + let query = if req.label_selector.is_empty() { + ObjectListQuery::Workspace(&workspace) + } else { + ObjectListQuery::WorkspaceSelector { + workspace: &workspace, + label_selector: &req.label_selector, + } + }; + let page = state + .store + .list_message_page::(query, after.as_ref(), pagination.page_size()) + .await + .map_err(|e| Status::internal(format!("list secrets failed: {e}")))?; + let next_page_token = pagination.next_object_token(page.next_cursor.as_ref()); + Ok(Response::new(ListSecretsResponse { + secrets: page + .messages + .iter() + .filter(|stored| !is_tombstoned(stored)) + .map(public_secret) + .collect(), + next_page_token, + })) +} + +pub(super) async fn handle_update_secret( + state: &Arc, + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let req = request.into_inner(); + if req.name.is_empty() { + return Err(Status::invalid_argument("name is required")); + } + let workspace = authorized_workspace( + state, + &principal, + req.workspace_scope.as_ref(), + MinWorkspaceRole::Admin, + ) + .await? + .name; + if let Some(value) = req.value.as_deref() { + validate_value(value)?; + } + if let Some(description) = req.description.as_deref() { + validate_description(description)?; + } + if let Some(labels) = req.labels.as_ref() { + validate_labels(&labels.labels)?; + } + + let current = get_live_secret(&state.store, &workspace, &req.name) + .await? + .ok_or_else(|| Status::not_found("secret not found"))?; + let current_version = current.get_resource_version(); + if let Some(expected) = req.expected_resource_version + && expected != current_version + { + return Err(Status::failed_precondition(format!( + "secret resource version is {current_version}, not {expected}" + ))); + } + if req.value.is_none() && req.description.is_none() && req.labels.is_none() { + return Ok(Response::new(SecretResponse { + secret: Some(public_secret(¤t)), + })); + } + + let mut updated = current.clone(); + let staged = if let Some(value) = req.value.as_deref() { + ensure_credential_storage(state)?; + let handle = stage_value(state, &workspace, current.object_id(), value).await?; + if let Some(previous) = updated.value_handle.replace(handle.clone()) { + updated.pending_handle_deletions.push(previous); + } + updated.value_update_time = + openshell_core::time::timestamp_from_millis(current_time_ms()).ok(); + Some(handle) + } else { + None + }; + if let Some(description) = req.description { + updated.description = description; + } + if let (Some(labels), Some(metadata)) = (req.labels, updated.metadata.as_mut()) { + metadata.labels = labels.labels; + } + + let labels = labels_json(&updated)?; + let write = state + .store + .put_if( + StoredWorkspaceSecret::object_type(), + updated.object_id(), + updated.object_name(), + &workspace, + &updated.encode_to_vec(), + labels.as_deref(), + WriteCondition::MatchResourceVersion(current_version), + ) + .await; + let write = match write { + Ok(write) => write, + Err(err) => { + if let Some(handle) = staged { + discard_staged_value(state, &workspace, current.object_id(), handle).await; + } + return Err(match err { + PersistenceError::Conflict { .. } => { + Status::aborted("secret was concurrently modified; retry the update") + } + err => Status::internal(format!("persist secret failed: {err}")), + }); + } + }; + if let Some(metadata) = updated.metadata.as_mut() { + metadata.resource_version = write.resource_version; + } + // Delete the superseded value now so a racing mint re-reads the record and + // uses the new value. Report the record as it ends up, so the returned + // resource version is current. + if !updated.pending_handle_deletions.is_empty() { + match drain_pending_deletions(state, updated.clone()).await { + Ok(drained) => updated = drained, + Err(err) => { + warn!(error = %err, "workspace secret handle cleanup deferred to the sweep"); + } + } + } + Ok(Response::new(SecretResponse { + secret: Some(public_secret(&updated)), + })) +} + +pub(super) async fn handle_delete_secret( + state: &Arc, + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let req = request.into_inner(); + if req.name.is_empty() { + return Err(Status::invalid_argument("name is required")); + } + let workspace = authorized_workspace( + state, + &principal, + req.workspace_scope.as_ref(), + MinWorkspaceRole::Admin, + ) + .await? + .name; + let stored = state + .store + .get_message_by_name::(&workspace, &req.name) + .await + .map_err(|e| Status::internal(format!("fetch secret failed: {e}")))?; + let Some(stored) = stored else { + return Ok(Response::new(DeleteSecretResponse { + outcome: super::deletion_outcome(false, req.allow_missing, "secret")?, + })); + }; + let tombstoned = tombstone(state, stored).await?; + finish_deletion(state, tombstoned).await?; + Ok(Response::new(DeleteSecretResponse { + outcome: DeletionOutcome::Completed.into(), + })) +} + +/// Mark a secret deleted and queue its value handle for backend deletion. A +/// tombstoned secret is invisible to reads and cannot be referenced. +async fn tombstone( + state: &ServerState, + mut stored: StoredWorkspaceSecret, +) -> Result { + if is_tombstoned(&stored) { + return Ok(stored); + } + let version = stored.get_resource_version(); + if let Some(metadata) = stored.metadata.as_mut() { + metadata.deletion_time = + openshell_core::time::timestamp_from_millis(current_time_ms()).ok(); + } + if let Some(handle) = stored.value_handle.take() { + stored.pending_handle_deletions.push(handle); + } + let write = state + .store + .put_if( + StoredWorkspaceSecret::object_type(), + stored.object_id(), + stored.object_name(), + stored.object_workspace(), + &stored.encode_to_vec(), + labels_json(&stored)?.as_deref(), + WriteCondition::MatchResourceVersion(version), + ) + .await + .map_err(|err| match err { + PersistenceError::Conflict { .. } => { + Status::aborted("secret was concurrently modified; retry the delete") + } + err => Status::internal(format!("tombstone secret failed: {err}")), + })?; + if let Some(metadata) = stored.metadata.as_mut() { + metadata.resource_version = write.resource_version; + } + Ok(stored) +} + +/// Delete a tombstoned secret's backend handles, then its record. +async fn finish_deletion(state: &ServerState, stored: StoredWorkspaceSecret) -> Result<(), Status> { + let stored = drain_pending_deletions(state, stored).await?; + state + .store + .delete_if( + StoredWorkspaceSecret::object_type(), + stored.object_id(), + stored.get_resource_version(), + ) + .await + .map_err(|e| Status::internal(format!("delete secret failed: {e}")))?; + Ok(()) +} + +/// Delete superseded backend handles, then clear them from the record. A +/// failure leaves the record unchanged so the sweep retries it. +async fn drain_pending_deletions( + state: &ServerState, + mut stored: StoredWorkspaceSecret, +) -> Result { + if stored.pending_handle_deletions.is_empty() { + return Ok(stored); + } + for handle in &stored.pending_handle_deletions { + state + .credentials + .delete_secrets(owner(&stored), &value_handles(handle.clone())) + .await?; + } + stored.pending_handle_deletions.clear(); + let write = state + .store + .put_if( + StoredWorkspaceSecret::object_type(), + stored.object_id(), + stored.object_name(), + stored.object_workspace(), + &stored.encode_to_vec(), + labels_json(&stored)?.as_deref(), + WriteCondition::MatchResourceVersion(stored.get_resource_version()), + ) + .await + .map_err(|err| match err { + PersistenceError::Conflict { .. } => { + Status::aborted("secret was concurrently modified during cleanup") + } + err => Status::internal(format!("persist secret cleanup failed: {err}")), + })?; + if let Some(metadata) = stored.metadata.as_mut() { + metadata.resource_version = write.resource_version; + } + Ok(stored) +} + +/// Retry backend deletions and tombstone removal that an earlier request could +/// not finish. Runs on the gateway's periodic maintenance tick. +pub async fn sweep_workspace_secrets(state: &ServerState) -> Result { + let records = state + .store + .list_by_type(StoredWorkspaceSecret::object_type(), SWEEP_LIMIT, 0) + .await + .map_err(|e| Status::internal(format!("list secrets failed: {e}")))?; + let mut finished = 0; + for record in records { + let mut stored = StoredWorkspaceSecret::decode(record.payload.as_slice()) + .map_err(|e| Status::internal(format!("decode secret failed: {e}")))?; + stored + .metadata + .get_or_insert_with(ObjectMeta::default) + .resource_version = record.resource_version; + if !is_tombstoned(&stored) && stored.pending_handle_deletions.is_empty() { + continue; + } + let result = if is_tombstoned(&stored) { + finish_deletion(state, stored).await + } else { + drain_pending_deletions(state, stored).await.map(|_| ()) + }; + match result { + Ok(()) => finished += 1, + Err(err) => warn!(error = %err, "workspace secret cleanup will be retried"), + } + } + Ok(finished) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::grpc::test_support::{authed_request, test_server_state}; + use openshell_core::proto::{SecretLabels, workspace_selector}; + use tonic::Code; + + type Mutate = fn(&mut CreateSecretRequest); + + async fn enabled_state() -> Arc { + let mut state = test_server_state().await; + Arc::get_mut(&mut state) + .expect("test state is not shared yet") + .config + .enable_workspace_secrets = true; + state + } + + fn create_request(name: &str, value: &str) -> CreateSecretRequest { + CreateSecretRequest { + workspace_scope: Some(workspace_selector("default".to_string())), + name: name.to_string(), + description: "shared OAuth client secret".to_string(), + value: value.to_string(), + allowed_endpoints: vec!["HTTPS://Login.Example.com:443/oauth/token".to_string()], + labels: HashMap::from([("team".to_string(), "platform".to_string())]), + request_id: String::new(), + } + } + + async fn create(state: &Arc, name: &str, value: &str) -> Secret { + handle_create_secret(state, authed_request(create_request(name, value))) + .await + .expect("create secret") + .into_inner() + .secret + .expect("secret") + } + + async fn stored(state: &Arc, name: &str) -> Option { + state + .store + .get_message_by_name::("default", name) + .await + .unwrap() + } + + async fn resolve_value( + state: &Arc, + stored: &StoredWorkspaceSecret, + handle: &CredentialHandle, + ) -> Result { + let resolved = state + .credentials + .resolve_secrets(owner(stored), &value_handles(handle.clone()), 0) + .await?; + resolved + .values + .get(SECRET_VALUE_KEY) + .cloned() + .ok_or_else(|| Status::not_found("value")) + } + + #[tokio::test] + async fn create_requires_operator_activation() { + let state = test_server_state().await; + let err = handle_create_secret(&state, authed_request(create_request("acme", "s3cret"))) + .await + .unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + assert!(err.message().contains("enable_workspace_secrets")); + assert!(stored(&state, "acme").await.is_none()); + } + + #[tokio::test] + async fn lifecycle_never_returns_the_value_and_rotates_handles() { + let state = enabled_state().await; + let created = create(&state, "acme", "first-value").await; + let metadata = created.metadata.as_ref().unwrap(); + assert_eq!(metadata.name, "acme"); + assert_eq!(metadata.workspace, "default"); + assert_ne!(metadata.resource_version, 0); + assert_eq!( + created.allowed_endpoints, + vec!["https://login.example.com/oauth/token".to_string()] + ); + let encoded = format!("{created:?}"); + assert!(!encoded.contains("first-value")); + + let record = stored(&state, "acme").await.unwrap(); + let first_handle = record.value_handle.clone().unwrap(); + assert_eq!( + resolve_value(&state, &record, &first_handle).await.unwrap(), + "first-value" + ); + + let fetched = handle_get_secret( + &state, + authed_request(GetSecretRequest { + workspace_scope: Some(workspace_selector("default".to_string())), + name: "acme".to_string(), + }), + ) + .await + .unwrap() + .into_inner() + .secret + .unwrap(); + assert_eq!(fetched, created); + + let updated = handle_update_secret( + &state, + authed_request(UpdateSecretRequest { + workspace_scope: Some(workspace_selector("default".to_string())), + name: "acme".to_string(), + value: Some("second-value".to_string()), + description: None, + labels: Some(SecretLabels { + labels: HashMap::new(), + }), + expected_resource_version: Some(metadata.resource_version), + request_id: String::new(), + }), + ) + .await + .unwrap() + .into_inner() + .secret + .unwrap(); + let record = stored(&state, "acme").await.unwrap(); + assert_eq!( + updated.metadata.as_ref().unwrap().resource_version, + record.get_resource_version(), + "the response reports the record after handle cleanup" + ); + assert!(record.pending_handle_deletions.is_empty()); + assert!(updated.metadata.as_ref().unwrap().labels.is_empty()); + assert_eq!(updated.description, "shared OAuth client secret"); + let second_handle = record.value_handle.clone().unwrap(); + assert_ne!(first_handle, second_handle); + assert_eq!( + resolve_value(&state, &record, &second_handle) + .await + .unwrap(), + "second-value" + ); + assert!( + resolve_value(&state, &record, &first_handle).await.is_err(), + "the superseded value is deleted from credential storage" + ); + + let listed = handle_list_secrets( + &state, + authed_request(ListSecretsRequest { + workspace_scope: Some(workspace_selector("default".to_string())), + page_size: 0, + page_token: String::new(), + label_selector: String::new(), + }), + ) + .await + .unwrap() + .into_inner(); + assert_eq!(listed.secrets, vec![updated]); + + let delete = |allow_missing| { + handle_delete_secret( + &state, + authed_request(DeleteSecretRequest { + workspace_scope: Some(workspace_selector("default".to_string())), + name: "acme".to_string(), + allow_missing, + request_id: String::new(), + }), + ) + }; + assert_eq!( + delete(false).await.unwrap().into_inner().outcome(), + DeletionOutcome::Completed + ); + assert!(stored(&state, "acme").await.is_none()); + assert!( + resolve_value(&state, &record, &second_handle) + .await + .is_err() + ); + assert_eq!(delete(false).await.unwrap_err().code(), Code::NotFound); + assert_eq!( + delete(true).await.unwrap().into_inner().outcome(), + DeletionOutcome::AlreadyAbsent + ); + } + + #[tokio::test] + async fn create_rejects_invalid_input_before_storing_anything() { + let state = enabled_state().await; + let cases: [(&str, Mutate); 5] = [ + ("empty value", |req| req.value.clear()), + ("no endpoints", |req| req.allowed_endpoints.clear()), + ("non-loopback http", |req| { + req.allowed_endpoints = vec!["http://login.example.com/token".to_string()]; + }), + ("query string", |req| { + req.allowed_endpoints = vec!["https://login.example.com/token?a=b".to_string()]; + }), + ("invalid name", |req| req.name = "Not_A_Label".to_string()), + ]; + for (case, mutate) in cases { + let mut request = create_request("acme", "value"); + mutate(&mut request); + let err = handle_create_secret(&state, authed_request(request)) + .await + .unwrap_err(); + assert_eq!(err.code(), Code::InvalidArgument, "{case}"); + } + assert!(stored(&state, "acme").await.is_none()); + } + + #[tokio::test] + async fn duplicate_names_are_rejected() { + let state = enabled_state().await; + create(&state, "acme", "value").await; + let err = handle_create_secret(&state, authed_request(create_request("acme", "other"))) + .await + .unwrap_err(); + assert_eq!(err.code(), Code::AlreadyExists); + } + + #[tokio::test] + async fn update_rejects_stale_versions_and_empty_values() { + let state = enabled_state().await; + let created = create(&state, "acme", "value").await; + let version = created.metadata.unwrap().resource_version; + let update = |value: Option<&str>, expected| { + handle_update_secret( + &state, + authed_request(UpdateSecretRequest { + workspace_scope: Some(workspace_selector("default".to_string())), + name: "acme".to_string(), + value: value.map(str::to_string), + description: None, + labels: None, + expected_resource_version: expected, + request_id: String::new(), + }), + ) + }; + let err = update(Some("next"), Some(version + 10)).await.unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + let err = update(Some(""), None).await.unwrap_err(); + assert_eq!(err.code(), Code::InvalidArgument); + let unchanged = update(None, None) + .await + .unwrap() + .into_inner() + .secret + .unwrap(); + assert_eq!(unchanged.metadata.unwrap().resource_version, version); + } + + #[tokio::test] + async fn sweep_finishes_an_interrupted_deletion() { + let state = enabled_state().await; + create(&state, "acme", "value").await; + let record = stored(&state, "acme").await.unwrap(); + let handle = record.value_handle.clone().unwrap(); + let tombstoned = tombstone(&state, record).await.unwrap(); + assert!(tombstoned.value_handle.is_none()); + assert_eq!(tombstoned.pending_handle_deletions, vec![handle.clone()]); + + // A tombstoned secret is invisible to reads. + assert!( + get_live_secret(&state.store, "default", "acme") + .await + .unwrap() + .is_none() + ); + + assert_eq!(sweep_workspace_secrets(&state).await.unwrap(), 1); + assert!(stored(&state, "acme").await.is_none()); + assert!(resolve_value(&state, &tombstoned, &handle).await.is_err()); + } +} diff --git a/crates/openshell-server/src/grpc/workspace.rs b/crates/openshell-server/src/grpc/workspace.rs index 9f3be64703..c51d378d16 100644 --- a/crates/openshell-server/src/grpc/workspace.rs +++ b/crates/openshell-server/src/grpc/workspace.rs @@ -403,6 +403,10 @@ pub(super) async fn handle_delete_workspace( StoredProviderCredentialRefreshState::object_type(), "credential refresh state", ), + ( + crate::storage_proto::StoredWorkspaceSecret::object_type(), + "secret", + ), ] { let records = state .store @@ -992,6 +996,54 @@ mod tests { ); } + #[tokio::test] + async fn delete_workspace_blocked_by_workspace_secret() { + let state = test_server_state().await; + + handle_create_workspace( + &state, + Request::new(CreateWorkspaceRequest { + request_id: String::new(), + name: "secretive".to_string(), + labels: HashMap::new(), + }), + ) + .await + .unwrap(); + + let secret = crate::storage_proto::StoredWorkspaceSecret { + metadata: Some(ObjectMeta { + id: "secret-1".to_string(), + name: "acme".to_string(), + created_time: openshell_core::time::timestamp_from_millis(1_000_000).ok(), + labels: HashMap::new(), + annotations: HashMap::new(), + resource_version: 0, + workspace: "secretive".to_string(), + deletion_time: None, + }), + ..Default::default() + }; + state.store.put_message(&secret).await.unwrap(); + + let err = handle_delete_workspace( + &state, + Request::new(DeleteWorkspaceRequest { + request_id: String::new(), + allow_missing: false, + name: "secretive".to_string(), + }), + ) + .await + .unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + assert!( + err.message().contains("secret"), + "error should name secret as blocker: {}", + err.message() + ); + } + #[tokio::test] async fn delete_workspace_blocked_by_provider_profiles() { let state = test_server_state().await; diff --git a/crates/openshell-server/src/persistence/image_activation.rs b/crates/openshell-server/src/persistence/image_activation.rs index e7024319b0..3d721dbd31 100644 --- a/crates/openshell-server/src/persistence/image_activation.rs +++ b/crates/openshell-server/src/persistence/image_activation.rs @@ -11,7 +11,7 @@ //! an accepted image mutation from a subsequent legacy Start. use super::{ObjectRecord, PersistenceError, PersistenceResult, Store}; -use crate::gateway_members::{MEMBER_TTL, SANDBOX_UPDATE_PROTOCOL}; +use crate::gateway_members::{MEMBER_TTL, SANDBOX_UPDATE_PROTOCOL, WORKSPACE_SECRETS_PROTOCOL}; use openshell_core::time::now_ms; const MEMBER_OBJECT_TYPE: &str = "gateway_member"; @@ -39,14 +39,61 @@ impl Store { .await?; ensure_live_members_compatible(&records, now_ms()) } + + /// Reject workspace-secret writes while an observable live gateway lacks + /// workspace-secret support. Same contract as image-update admission: the + /// caller enforces the default-off activation flag, `SQLite` relies on its + /// single-gateway contract, and the snapshot is not a fence. + pub async fn ensure_workspace_secrets_deployment_compatible(&self) -> PersistenceResult<()> { + let Self::Postgres(_) = self else { + return Ok(()); + }; + let records = self + .list_by_type(MEMBER_OBJECT_TYPE, MEMBER_LIMIT + 1, 0) + .await?; + ensure_live_members_advertise(&records, now_ms(), &WORKSPACE_SECRETS_CAPABILITY) + } +} + +/// A protocol a live gateway must advertise in its membership record. +struct MemberCapability { + field: &'static str, + protocol: u32, + feature: &'static str, + requirement: &'static str, + protocol_label: &'static str, } +const SANDBOX_UPDATE_CAPABILITY: MemberCapability = MemberCapability { + field: "sandbox_update_protocol", + protocol: SANDBOX_UPDATE_PROTOCOL, + feature: "image-update", + requirement: "sandbox updates require", + protocol_label: "sandbox-update", +}; + +const WORKSPACE_SECRETS_CAPABILITY: MemberCapability = MemberCapability { + field: "workspace_secrets_protocol", + protocol: WORKSPACE_SECRETS_PROTOCOL, + feature: "workspace-secrets", + requirement: "workspace secrets require", + protocol_label: "workspace-secrets", +}; + fn ensure_live_members_compatible(records: &[ObjectRecord], now: i64) -> PersistenceResult<()> { + ensure_live_members_advertise(records, now, &SANDBOX_UPDATE_CAPABILITY) +} + +fn ensure_live_members_advertise( + records: &[ObjectRecord], + now: i64, + capability: &MemberCapability, +) -> PersistenceResult<()> { + let feature = capability.feature; if records.len() > MEMBER_LIMIT as usize { - return Err(PersistenceError::Config( - "cannot establish image-update deployment compatibility: gateway membership listing exceeds its limit" - .into(), - )); + return Err(PersistenceError::Config(format!( + "cannot establish {feature} deployment compatibility: gateway membership listing exceeds its limit" + ))); } let ttl_ms = i64::try_from(MEMBER_TTL.as_millis()).unwrap_or(i64::MAX); for record in records { @@ -58,20 +105,20 @@ fn ensure_live_members_compatible(records: &[ObjectRecord], now: i64) -> Persist } let member: serde_json::Map = serde_json::from_slice(&record.payload).map_err(|_| { - PersistenceError::Config( - "cannot establish image-update deployment compatibility: a live gateway membership record is not a valid JSON object" - .into(), - ) + PersistenceError::Config(format!( + "cannot establish {feature} deployment compatibility: a live gateway membership record is not a valid JSON object" + )) })?; let replica_id = member.get("replica_id").and_then(serde_json::Value::as_str); let protocol = member - .get("sandbox_update_protocol") + .get(capability.field) .and_then(serde_json::Value::as_u64); if replica_id.is_none_or(|id| id.trim().is_empty()) - || protocol != Some(u64::from(SANDBOX_UPDATE_PROTOCOL)) + || protocol != Some(u64::from(capability.protocol)) { return Err(PersistenceError::Config(format!( - "sandbox updates require every live gateway membership record to advertise sandbox-update protocol {SANDBOX_UPDATE_PROTOCOL} and a nonempty replica id" + "{} every live gateway membership record to advertise {} protocol {} and a nonempty replica id", + capability.requirement, capability.protocol_label, capability.protocol ))); } } @@ -201,6 +248,23 @@ mod tests { assert!(ensure_live_members_compatible(&[record(b"legacy", 200)], 100).is_err()); } + #[test] + fn workspace_secrets_require_their_own_protocol() { + let check = |payload: &[u8]| { + ensure_live_members_advertise( + &[record(payload, 100)], + 100, + &WORKSPACE_SECRETS_CAPABILITY, + ) + }; + assert!(check(br#"{"replica_id":"gw-0","workspace_secrets_protocol":1}"#).is_ok()); + // A gateway that only knows sandbox updates predates workspace secrets. + let err = check(br#"{"replica_id":"gw-0","sandbox_update_protocol":1}"#).unwrap_err(); + assert!(err.to_string().contains("workspace secrets require")); + assert!(check(br#"{"replica_id":"gw-0","workspace_secrets_protocol":2}"#).is_err()); + assert!(check(br#"{"workspace_secrets_protocol":1}"#).is_err()); + } + #[test] fn any_incompatible_live_member_rejects_the_snapshot() { let compatible = record(br#"{"replica_id":"gw-0","sandbox_update_protocol":1}"#, 100); diff --git a/crates/openshell-server/src/provider_refresh.rs b/crates/openshell-server/src/provider_refresh.rs index 2ed150652b..4c7a6c5970 100644 --- a/crates/openshell-server/src/provider_refresh.rs +++ b/crates/openshell-server/src/provider_refresh.rs @@ -319,6 +319,9 @@ pub fn spawn_refresh_worker(state: Arc, interval: Duration) warn!(error = %err, "provider credential refresh worker tick failed"); } } + if let Err(err) = crate::grpc::secret::sweep_workspace_secrets(&state).await { + warn!(error = %err, "workspace secret cleanup sweep failed"); + } } }); } diff --git a/crates/openshell-server/src/storage_proto.rs b/crates/openshell-server/src/storage_proto.rs index 723cbed928..64a5cb03b7 100644 --- a/crates/openshell-server/src/storage_proto.rs +++ b/crates/openshell-server/src/storage_proto.rs @@ -20,6 +20,11 @@ mod sandbox_image { } pub(crate) use sandbox_image::{StoredSandboxImageState, StoredSandboxImageUpdate}; +mod workspace_secret { + include!(concat!(env!("OUT_DIR"), "/openshell.storage.secret.v1.rs")); +} +pub(crate) use workspace_secret::StoredWorkspaceSecret; + pub(crate) const STORAGE_FILE_DESCRIPTOR_SET: &[u8] = include_bytes!(concat!(env!("OUT_DIR"), "/storage_descriptor.bin")); @@ -112,6 +117,48 @@ impl ObjectWorkspace for StoredProviderCredentialRefreshStateV2 { } } +impl ObjectId for StoredWorkspaceSecret { + fn object_id(&self) -> &str { + self.metadata.as_ref().map_or("", |m| m.id.as_str()) + } +} + +impl ObjectName for StoredWorkspaceSecret { + fn object_name(&self) -> &str { + self.metadata.as_ref().map_or("", |m| m.name.as_str()) + } +} + +impl ObjectLabels for StoredWorkspaceSecret { + fn object_labels(&self) -> Option> { + self.metadata.as_ref().map(|m| m.labels.clone()) + } +} + +impl SetResourceVersion for StoredWorkspaceSecret { + fn set_resource_version(&mut self, version: u64) { + if let Some(meta) = self.metadata.as_mut() { + meta.resource_version = version; + } + } +} + +impl GetResourceVersion for StoredWorkspaceSecret { + fn get_resource_version(&self) -> u64 { + self.metadata.as_ref().map_or(0, |m| m.resource_version) + } +} + +impl ObjectWorkspace for StoredWorkspaceSecret { + fn object_workspace(&self) -> &str { + self.metadata.as_ref().map_or("", |m| m.workspace.as_str()) + } + + fn requires_workspace() -> bool { + true + } +} + #[cfg(test)] mod tests { use super::*; @@ -179,9 +226,9 @@ mod tests { // updates use a separate capability/service and add one classified RPC. // Discovery RPCs move to ServiceDiscovery; message and durable closures are unchanged. const PUBLIC_RPC_SCHEMA_SHA256: &str = - "daa132037ee47c7a46b08dfab9bab379961bc06ff3e9ebbb329809b724f2a4f6"; + "76d166f76b8aa0abc7d7611b642e5f499b762c746b37f7f183e401cb0ea23685"; const DURABLE_SCHEMA_SHA256: &str = - "cb15f8f79a855064a43fe8c6cd908bf07eca6604e2bf8ab7d2de0ef33e97da4e"; + "b6a1de55ca3bdf2ccbf88a32f57e87db84b775c48fcd5f9fcf7811eae8d67c70"; const PUBLIC_DURABLE_OVERLAP_SHA256: &str = "c0c3ea17f105aaa6d843c466fd4240451c2105dce7675384935d4aaecdac8abf"; // Encoded with the schema that omitted stable_placeholder. Keep these @@ -245,6 +292,7 @@ mod tests { ".openshell.storage.v1.StoredProviderCredentialRefreshStateV2", ".openshell.storage.v1.StoredProviderProfile", ".openshell.storage.image.v1.StoredSandboxImageState", + ".openshell.storage.secret.v1.StoredWorkspaceSecret", ".openshell.v1.Sandbox", ".openshell.v1.SandboxWorkloadTemplate", ".openshell.v1.ServiceEndpoint", @@ -655,7 +703,7 @@ mod tests { } assert_eq!( compiled_method_count, - 113 + PROVIDER_READINESS_RPC_SIGNATURES.len() + 118 + PROVIDER_READINESS_RPC_SIGNATURES.len() + PEER_OWNER_RPC_SIGNATURES.len() + PEER_CONFIG_RPC_SIGNATURES.len() + isolation_methods.len() @@ -665,7 +713,7 @@ mod tests { ); assert_eq!( methods.len(), - 83 + PROVIDER_READINESS_RPC_SIGNATURES.len() + 88 + PROVIDER_READINESS_RPC_SIGNATURES.len() + PEER_OWNER_RPC_SIGNATURES.len() + PEER_CONFIG_RPC_SIGNATURES.len(), "inventory every public gateway RPC" @@ -675,7 +723,7 @@ mod tests { .iter() .filter(|method| method.starts_with("openshell.v1.OpenShell/")) .count(), - 80 + PROVIDER_READINESS_RPC_SIGNATURES.len() + 85 + PROVIDER_READINESS_RPC_SIGNATURES.len() + PEER_OWNER_RPC_SIGNATURES.len() + PEER_CONFIG_RPC_SIGNATURES.len() ); @@ -726,8 +774,8 @@ mod tests { overlap_hash.as_str(), ), ( - (344, 31), - (100, 22), + (356, 31), + (101, 22), (86, 22), PUBLIC_RPC_SCHEMA_SHA256, DURABLE_SCHEMA_SHA256, diff --git a/crates/openshell-server/tests/common/mod.rs b/crates/openshell-server/tests/common/mod.rs index dd91664d39..13628249dd 100644 --- a/crates/openshell-server/tests/common/mod.rs +++ b/crates/openshell-server/tests/common/mod.rs @@ -414,6 +414,41 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn create_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn get_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn list_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn update_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + + async fn delete_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn update_provider( &self, _request: tonic::Request, diff --git a/crates/openshell-server/tests/supervisor_relay_integration.rs b/crates/openshell-server/tests/supervisor_relay_integration.rs index ceb2fa6bf8..1e6cc0c96d 100644 --- a/crates/openshell-server/tests/supervisor_relay_integration.rs +++ b/crates/openshell-server/tests/supervisor_relay_integration.rs @@ -441,6 +441,41 @@ impl OpenShell for RelayGateway { ) -> Result, Status> { Err(Status::unimplemented("unused")) } + + async fn create_secret( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn get_secret( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn list_secrets( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn update_secret( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + + async fn delete_secret( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } async fn get_provider_refresh_status( &self, _: tonic::Request, diff --git a/proto/openshell.proto b/proto/openshell.proto index 198990a364..fb5fe46994 100644 --- a/proto/openshell.proto +++ b/proto/openshell.proto @@ -450,6 +450,53 @@ service OpenShell { }; } + // Create a workspace secret. Secrets are credential minting material only: + // OpenShell never injects them into sandboxes, and Get and List never + // return their values. + rpc CreateSecret(CreateSecretRequest) returns (SecretResponse) { + option (openshell.options.v1.authorization) = { + auth_mode: "bearer" + scope: "secret:write" + workspace_role: "admin" + }; + } + + // Fetch workspace secret metadata by name. The value is never returned. + rpc GetSecret(GetSecretRequest) returns (SecretResponse) { + option (openshell.options.v1.authorization) = { + auth_mode: "bearer" + scope: "secret:read" + workspace_role: "user" + }; + } + + // List workspace secret metadata. Values are never returned. + rpc ListSecrets(ListSecretsRequest) returns (ListSecretsResponse) { + option (openshell.options.v1.authorization) = { + auth_mode: "bearer" + scope: "secret:read" + workspace_role: "user" + }; + } + + // Replace a workspace secret's value, description, or labels. + rpc UpdateSecret(UpdateSecretRequest) returns (SecretResponse) { + option (openshell.options.v1.authorization) = { + auth_mode: "bearer" + scope: "secret:write" + workspace_role: "admin" + }; + } + + // Delete a workspace secret by name. + rpc DeleteSecret(DeleteSecretRequest) returns (DeleteSecretResponse) { + option (openshell.options.v1.authorization) = { + auth_mode: "bearer" + scope: "secret:write" + workspace_role: "admin" + }; + } + // Get sandbox settings by id (called by users, and by supervisors at startup // and in their poll loop). A supervisor whose session pushes configuration // receives later updates over ConnectSupervisor and stops polling. @@ -2778,6 +2825,100 @@ message DeleteProviderProfileResponse { DeletionOutcome outcome = 2; } +// A workspace-scoped value used only as credential minting material, such as +// an OAuth client secret shared by many providers. OpenShell never injects a +// secret into sandboxes and never returns its value from Get or List. +message Secret { + openshell.datamodel.v1.ObjectMeta metadata = 1; + string description = 2; + // Token endpoints this value may be sent to, normalized. Fixed at creation. + repeated string allowed_endpoints = 3; + // Last time the value was written. + google.protobuf.Timestamp value_update_time = 4; +} + +message CreateSecretRequest { + // Workspace scope. Only a named workspace selection is accepted. + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string name = 2; + string description = 3; + // Required and non-empty. + string value = 4 [(openshell.options.v1.secret) = true]; + // Required. Absolute https URLs, or http URLs for loopback hosts. Compared + // exactly after normalization. Cannot be changed after creation. + repeated string allowed_endpoints = 5; + map labels = 6; + // Optional nonzero UUID for durable at-most-once admission. Successful results + // can be replayed for 24 hours; see the API errors and retries reference. + string request_id = 7; +} + +message GetSecretRequest { + // Workspace scope. Only a named workspace selection is accepted. + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string name = 2; +} + +message ListSecretsRequest { + // Workspace scope. Only a named workspace selection is accepted. + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + // The maximum number of secrets to return. Zero uses 100. Values above 1000 + // are coerced to 1000; negative values are invalid. + int32 page_size = 2; + // Token from a previous ListSecrets response. All other request parameters + // except page_size must match the request that produced it. + string page_token = 3; + // Optional label selector in key=value comma-separated form. + string label_selector = 4; +} + +// Presence selects replacement, including an explicitly empty map. +message SecretLabels { + map labels = 1; +} + +message UpdateSecretRequest { + // Workspace scope. Only a named workspace selection is accepted. + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string name = 2; + // Absent leaves the value unchanged. Present must be non-empty. + optional string value = 3 [(openshell.options.v1.secret) = true]; + // Absent leaves the description unchanged. + optional string description = 4; + // Absent leaves labels unchanged. Present replaces the full set. + SecretLabels labels = 5; + // Rejects the update unless it matches the current resource version. + optional uint64 expected_resource_version = 6; + // Optional nonzero UUID for durable at-most-once admission. Successful results + // can be replayed for 24 hours; see the API errors and retries reference. + string request_id = 7; +} + +message DeleteSecretRequest { + // Workspace scope. Only a named workspace selection is accepted. + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string name = 2; + // Succeed with ALREADY_ABSENT if the target is missing. Authorization and + // parent-workspace checks still apply. + bool allow_missing = 3; + // Optional nonzero UUID. Same ID and payload replay success for 24 hours. + string request_id = 4; +} + +message SecretResponse { + Secret secret = 1; +} + +message ListSecretsResponse { + repeated Secret secrets = 1; + // Token for the next page. Empty when there are no subsequent pages. + string next_page_token = 2; +} + +message DeleteSecretResponse { + DeletionOutcome outcome = 1; +} + // Get sandbox provider environment request. message GetSandboxProviderEnvironmentRequest { // The sandbox ID. From 6970aa824a8656b1c3094ecf191fdaeecda292be Mon Sep 17 00:00:00 2001 From: Mrunal Patel Date: Wed, 18 Nov 2026 07:51:55 +0000 Subject: [PATCH 03/10] feat(server): reference workspace secrets from provider refresh Let ConfigureProviderRefresh take secret material from a workspace secret by name. The gateway binds each key to the secret's immutable ID in a refresh-state annotation, which older gateways preserve, and requires the key in secret_material_keys so those gateways reject the request and refuse to mint without it. Only secret material can be referenced, never a key the minter rotates per provider. The token endpoint a mint will call must be one the secret allows; the minter re-checks it before sending and refuses redirects for requests carrying a workspace secret. Mints read the secret's current value and re-read the record once if an update raced them. DeleteSecret refuses while a refresh references the secret, and the new InlineProviderRefreshSecrets RPC copies referenced values into provider-owned material without disturbing a rotated refresh token. Refs #5563 Signed-off-by: Mrunal Patel --- crates/openshell-cli/src/commands/provider.rs | 61 ++- crates/openshell-cli/src/main.rs | 35 ++ crates/openshell-cli/src/run.rs | 4 +- .../tests/ensure_providers_integration.rs | 7 + .../openshell-cli/tests/mtls_integration.rs | 7 + .../tests/provider_commands_integration.rs | 98 ++++ .../sandbox_create_lifecycle_integration.rs | 7 + .../sandbox_exec_streaming_integration.rs | 1 + .../sandbox_name_fallback_integration.rs | 7 + .../src/refresh/engine.rs | 278 +++++++++- .../src/refresh/minter.rs | 222 +++++++- .../src/refresh/mod.rs | 11 + .../src/refresh/secret_refs.rs | 157 ++++++ crates/openshell-sdk/tests/client_mock.rs | 7 + .../src/auth/audit/catalog.rs | 1 + crates/openshell-server/src/grpc/mod.rs | 12 +- .../src/grpc/mutation_replay/ordinary.rs | 9 +- crates/openshell-server/src/grpc/provider.rs | 352 +++++++++++- crates/openshell-server/src/grpc/secret.rs | 51 +- .../src/grpc/secret_refresh_tests.rs | 509 ++++++++++++++++++ .../openshell-server/src/provider_refresh.rs | 10 + crates/openshell-server/src/refresh_store.rs | 38 +- crates/openshell-server/src/storage_proto.rs | 10 +- crates/openshell-server/tests/common/mod.rs | 7 + .../tests/supervisor_relay_integration.rs | 7 + proto/openshell.proto | 38 +- 26 files changed, 1911 insertions(+), 35 deletions(-) create mode 100644 crates/openshell-credential-issuer/src/refresh/secret_refs.rs create mode 100644 crates/openshell-server/src/grpc/secret_refresh_tests.rs diff --git a/crates/openshell-cli/src/commands/provider.rs b/crates/openshell-cli/src/commands/provider.rs index efc3d1bb12..5ec34cc316 100644 --- a/crates/openshell-cli/src/commands/provider.rs +++ b/crates/openshell-cli/src/commands/provider.rs @@ -19,12 +19,12 @@ use openshell_core::proto::{ DeleteProviderProfileRequest, DeleteProviderRefreshRequest, DeleteProviderRequest, DetachSandboxProviderRequest, GetProviderProfileRequest, GetProviderRefreshStatusRequest, GetProviderRequest, GetSandboxRequest, ImportProviderProfilesRequest, - LintProviderProfilesRequest, ListProviderProfilesRequest, ListProvidersRequest, - ListSandboxProvidersRequest, Provider, ProviderCredentialRefreshRecoveryAction, - ProviderCredentialRefreshStatus, ProviderCredentialRefreshStrategy, - ProviderCredentialTokenGrantType, ProviderMutationKind, ProviderProfile, - ProviderProfileDiagnostic, ProviderProfileImportItem, RotateProviderCredentialRequest, - UpdateProviderProfilesRequest, UpdateProviderRequest, + InlineProviderRefreshSecretsRequest, LintProviderProfilesRequest, ListProviderProfilesRequest, + ListProvidersRequest, ListSandboxProvidersRequest, Provider, + ProviderCredentialRefreshRecoveryAction, ProviderCredentialRefreshStatus, + ProviderCredentialRefreshStrategy, ProviderCredentialTokenGrantType, ProviderMutationKind, + ProviderProfile, ProviderProfileDiagnostic, ProviderProfileImportItem, + RotateProviderCredentialRequest, UpdateProviderProfilesRequest, UpdateProviderRequest, }; use openshell_core::rpc_error::{ERROR_DOMAIN, decode_details}; use openshell_core::{ObjectId, ObjectName, ObjectWorkspace}; @@ -1291,6 +1291,7 @@ pub async fn provider_create_with_options(options: ProviderCreateOptions<'_>) -> workspace_scope: Some(openshell_core::proto::workspace_selector( (workspace).to_string(), )), + material_secrets: HashMap::new(), }) .await { @@ -1964,6 +1965,7 @@ pub struct ProviderRefreshConfigInput<'a> { pub material: &'a [String], pub secret_material_env: &'a [String], pub secret_material_keys: &'a [String], + pub material_secrets: &'a [String], pub credential_expires_at_ms: Option, } @@ -1989,6 +1991,20 @@ pub async fn provider_refresh_config( } material.insert(key, value); } + // Workspace-secret references are secret material too. Listing them in + // secret_material_keys makes gateways that predate workspace secrets reject + // the request instead of ignoring the references. + let material_secrets = parse_key_value_pairs(input.material_secrets, "--material-secret")?; + for key in material_secrets.keys() { + if material.contains_key(key) { + return Err(miette!( + "duplicate material key '{key}': supplied via both inline material and --material-secret" + )); + } + if !secret_material_keys.contains(key) { + secret_material_keys.push(key.clone()); + } + } let mut client = grpc_client(server, tls).await?; let status = client .configure_provider_refresh(ConfigureProviderRefreshRequest { @@ -1998,6 +2014,7 @@ pub async fn provider_refresh_config( strategy: strategy as i32, material, secret_material_keys, + material_secrets, expiration_time: input .credential_expires_at_ms .map(openshell_core::time::timestamp_from_millis) @@ -2062,6 +2079,37 @@ pub async fn provider_rotate( Ok(()) } +pub async fn provider_refresh_inline_secrets( + server: &str, + name: &str, + credential_key: &str, + workspace: &str, + tls: &TlsOptions, +) -> Result<()> { + let mut client = grpc_client(server, tls).await?; + let status = client + .inline_provider_refresh_secrets(InlineProviderRefreshSecretsRequest { + workspace_scope: Some(openshell_core::proto::workspace_selector( + workspace.to_string(), + )), + provider: name.to_string(), + credential_key: credential_key.to_string(), + request_id: String::new(), + }) + .await + .into_diagnostic()? + .into_inner() + .status + .ok_or_else(|| miette!("provider refresh status missing from response"))?; + println!( + "{} Copied workspace secrets into provider-owned material for {} {}", + "✓".green().bold(), + status.provider, + status.credential_key + ); + Ok(()) +} + pub async fn provider_refresh_delete( server: &str, name: &str, @@ -3477,6 +3525,7 @@ binaries: [/usr/bin/curl] failure_code: "oauth_rotated_refresh_token_handle_missing".to_string(), provider_error_subtype: "invalid_rapt".to_string(), last_error_time: openshell_core::time::timestamp_from_millis(1_767_225_000_000).ok(), + material_secrets: HashMap::new(), }); assert!(row.contains("my-graph")); diff --git a/crates/openshell-cli/src/main.rs b/crates/openshell-cli/src/main.rs index 8bcf8f00f3..767c1f5b06 100644 --- a/crates/openshell-cli/src/main.rs +++ b/crates/openshell-cli/src/main.rs @@ -1044,6 +1044,11 @@ enum ProviderRefreshCommands { #[arg(long = "secret-material-key", value_name = "KEY")] secret_material_keys: Vec, + /// Secret refresh material supplied by a workspace secret + /// (`KEY=SECRET`); `KEY` is auto-marked secret. + #[arg(long = "material-secret", value_name = "KEY=SECRET")] + material_secrets: Vec, + /// Expiry for the current credential. Accepts epoch milliseconds or RFC3339. #[arg( long = "credential-expires-at", @@ -1076,6 +1081,21 @@ enum ProviderRefreshCommands { #[arg(long = "credential-key")] credential_key: String, }, + + /// Copy referenced workspace secrets into provider-owned refresh material. + /// + /// Keeps the current grant, including a rotated refresh token. Run it + /// before deleting a secret or rolling back workspace secrets. + #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] + InlineSecrets { + /// Provider name. + #[arg(add = ArgValueCompleter::new(completers::complete_provider_names))] + name: String, + + /// Injectable credential key, for example `MS_GRAPH_ACCESS_TOKEN`. + #[arg(long = "credential-key")] + credential_key: String, + }, } /// Profile kinds exposed by the gateway's profile catalog. @@ -4021,6 +4041,7 @@ async fn run_async() -> Result<()> { material, secret_material_env, secret_material_keys, + material_secrets, credential_expires_at, } => { run::provider_refresh_config( @@ -4032,6 +4053,7 @@ async fn run_async() -> Result<()> { material: &material, secret_material_env: &secret_material_env, secret_material_keys: &secret_material_keys, + material_secrets: &material_secrets, credential_expires_at_ms: credential_expires_at, }, &cli.workspace, @@ -4065,6 +4087,19 @@ async fn run_async() -> Result<()> { ) .await?; } + ProviderRefreshCommands::InlineSecrets { + name, + credential_key, + } => { + run::provider_refresh_inline_secrets( + endpoint, + &name, + &credential_key, + &cli.workspace, + &tls, + ) + .await?; + } }, ProviderCommands::ListProfiles { output, global } => { // Normalize the legacy list command before execution so output, diff --git a/crates/openshell-cli/src/run.rs b/crates/openshell-cli/src/run.rs index 7cd4ec016b..9d12432ac9 100644 --- a/crates/openshell-cli/src/run.rs +++ b/crates/openshell-cli/src/run.rs @@ -30,8 +30,8 @@ pub use crate::commands::provider::{ provider_profile_describe, provider_profile_describe_text, provider_profile_export, provider_profile_export_text, provider_profile_import, provider_profile_lint, provider_profile_update, provider_refresh_config, provider_refresh_delete, - provider_refresh_status, provider_rotate, provider_update, sandbox_provider_attach, - sandbox_provider_detach, sandbox_provider_list, + provider_refresh_inline_secrets, provider_refresh_status, provider_rotate, provider_update, + sandbox_provider_attach, sandbox_provider_detach, sandbox_provider_list, }; pub use crate::commands::provider_readiness::{ProviderWaitOptions, sandbox_provider_status}; diff --git a/crates/openshell-cli/tests/ensure_providers_integration.rs b/crates/openshell-cli/tests/ensure_providers_integration.rs index f83c382692..dd4dc61a80 100644 --- a/crates/openshell-cli/tests/ensure_providers_integration.rs +++ b/crates/openshell-cli/tests/ensure_providers_integration.rs @@ -503,6 +503,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn inline_provider_refresh_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn update_provider( &self, request: tonic::Request, diff --git a/crates/openshell-cli/tests/mtls_integration.rs b/crates/openshell-cli/tests/mtls_integration.rs index 594eb37c95..10e38b2356 100644 --- a/crates/openshell-cli/tests/mtls_integration.rs +++ b/crates/openshell-cli/tests/mtls_integration.rs @@ -418,6 +418,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn inline_provider_refresh_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn update_provider( &self, _request: tonic::Request, diff --git a/crates/openshell-cli/tests/provider_commands_integration.rs b/crates/openshell-cli/tests/provider_commands_integration.rs index f4afad4fd9..f0d48676d4 100644 --- a/crates/openshell-cli/tests/provider_commands_integration.rs +++ b/crates/openshell-cli/tests/provider_commands_integration.rs @@ -127,6 +127,7 @@ enum ProviderRefreshRequestLog { credential_key: String, material: HashMap, secret_material_keys: Vec, + material_secrets: HashMap, expires_at_ms: Option, }, Rotate { @@ -1242,6 +1243,7 @@ impl OpenShell for TestOpenShell { credential_key: request.credential_key.clone(), material: request.material.clone(), secret_material_keys: request.secret_material_keys.clone(), + material_secrets: request.material_secrets.clone(), expires_at_ms: request .expiration_time .as_ref() @@ -1274,6 +1276,7 @@ impl OpenShell for TestOpenShell { failure_code: String::new(), provider_error_subtype: String::new(), last_error_time: None, + material_secrets: HashMap::new(), }; drop(providers); self.state @@ -1452,6 +1455,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn inline_provider_refresh_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + type WatchSandboxStream = tokio_stream::wrappers::ReceiverStream>; type ExecSandboxStream = @@ -3940,6 +3950,7 @@ async fn provider_refresh_cli_run_functions_wire_requests() { secret_material_env: &[], secret_material_keys: &["client_secret".to_string()], credential_expires_at_ms: Some(1_767_225_600_000), + material_secrets: &[], }, "default", &ts.tls, @@ -3983,6 +3994,7 @@ async fn provider_refresh_cli_run_functions_wire_requests() { credential_key: "MS_GRAPH_ACCESS_TOKEN".to_string(), material: HashMap::from([("tenant_id".to_string(), "tenant".to_string())]), secret_material_keys: vec!["client_secret".to_string()], + material_secrets: HashMap::new(), expires_at_ms: Some(1_767_225_600_000), }, ProviderRefreshRequestLog::Status { @@ -4036,6 +4048,7 @@ async fn provider_refresh_configure_reads_secret_material_from_env_off_argv() { ], secret_material_keys: &[], credential_expires_at_ms: None, + material_secrets: &[], }, "default", &ts.tls, @@ -4057,6 +4070,7 @@ async fn provider_refresh_configure_reads_secret_material_from_env_off_argv() { ("private_key".to_string(), "pem-from-env".to_string()), ]), secret_material_keys: vec!["private_key".to_string()], + material_secrets: HashMap::new(), expires_at_ms: None, }] ); @@ -4064,6 +4078,88 @@ async fn provider_refresh_configure_reads_secret_material_from_env_off_argv() { .await; } +#[tokio::test] +async fn provider_refresh_configure_sends_workspace_secret_references_as_secret_keys() { + let ts = run_server().await; + install_test_profile(&ts, "custom-chat", "GOOGLE_CHAT_ACCESS_TOKEN").await; + run::provider_create( + &ts.endpoint, + "alice-mcp", + "custom-chat", + false, + &["GOOGLE_CHAT_ACCESS_TOKEN=pending".to_string()], + false, + &[], + "default", + &ts.tls, + ) + .await + .expect("provider create"); + + run::provider_refresh_config( + &ts.endpoint, + run::ProviderRefreshConfigInput { + name: "alice-mcp", + credential_key: "GOOGLE_CHAT_ACCESS_TOKEN", + strategy: "oauth2_client_credentials", + material: &["client_id=client-id".to_string()], + secret_material_env: &[], + secret_material_keys: &[], + credential_expires_at_ms: None, + material_secrets: &["client_secret=acme-oauth-client".to_string()], + }, + "default", + &ts.tls, + ) + .await + .expect("provider refresh configure"); + + let requests = ts.state.refresh_requests.lock().await.clone(); + assert_eq!( + requests, + vec![ProviderRefreshRequestLog::Configure { + provider: "alice-mcp".to_string(), + credential_key: "GOOGLE_CHAT_ACCESS_TOKEN".to_string(), + material: HashMap::from([("client_id".to_string(), "client-id".to_string())]), + // Listed so gateways that predate workspace secrets reject the + // request rather than ignore the reference. + secret_material_keys: vec!["client_secret".to_string()], + material_secrets: HashMap::from([( + "client_secret".to_string(), + "acme-oauth-client".to_string() + )]), + expires_at_ms: None, + }] + ); +} + +#[tokio::test] +async fn provider_refresh_configure_rejects_key_supplied_inline_and_by_secret() { + let ts = run_server().await; + let err = run::provider_refresh_config( + &ts.endpoint, + run::ProviderRefreshConfigInput { + name: "alice-mcp", + credential_key: "GOOGLE_CHAT_ACCESS_TOKEN", + strategy: "oauth2_client_credentials", + material: &["client_secret=inline".to_string()], + secret_material_env: &[], + secret_material_keys: &[], + credential_expires_at_ms: None, + material_secrets: &["client_secret=acme-oauth-client".to_string()], + }, + "default", + &ts.tls, + ) + .await + .expect_err("a key cannot come from both inline material and a secret"); + assert!( + err.to_string() + .contains("duplicate material key 'client_secret'") + ); + assert!(ts.state.refresh_requests.lock().await.is_empty()); +} + #[tokio::test] async fn provider_refresh_configure_rejects_key_supplied_via_both_material_and_env() { let ts = run_server().await; @@ -4080,6 +4176,7 @@ async fn provider_refresh_configure_rejects_key_supplied_via_both_material_and_e secret_material_env: &["private_key=OPENSHELL_ITEST_SME_DUP_KEY".to_string()], secret_material_keys: &[], credential_expires_at_ms: None, + material_secrets: &[], }, "default", &ts.tls, @@ -4111,6 +4208,7 @@ async fn provider_refresh_configure_fails_closed_when_secret_material_env_is_uns secret_material_env: &["private_key=OPENSHELL_ITEST_SME_DEFINITELY_UNSET".to_string()], secret_material_keys: &[], credential_expires_at_ms: None, + material_secrets: &[], }, "default", &ts.tls, diff --git a/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs b/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs index 19e0cc09be..0fbe659e43 100644 --- a/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs +++ b/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs @@ -706,6 +706,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn inline_provider_refresh_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn update_provider( &self, _request: tonic::Request, diff --git a/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs b/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs index 0d7e693610..44f93d672a 100644 --- a/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs +++ b/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs @@ -266,6 +266,7 @@ mock_gateway! { list_secrets(proto::ListSecretsRequest) -> proto::ListSecretsResponse; update_secret(proto::UpdateSecretRequest) -> proto::SecretResponse; delete_secret(proto::DeleteSecretRequest) -> proto::DeleteSecretResponse; + inline_provider_refresh_secrets(proto::InlineProviderRefreshSecretsRequest) -> proto::InlineProviderRefreshSecretsResponse; get_sandbox_config(proto::GetSandboxConfigRequest) -> proto::GetSandboxConfigResponse; get_gateway_config(proto::GetGatewayConfigRequest) -> proto::GetGatewayConfigResponse; update_config(proto::UpdateConfigRequest) -> proto::UpdateConfigResponse; diff --git a/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs b/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs index 89c2723b81..55598ca08e 100644 --- a/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs +++ b/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs @@ -476,6 +476,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn inline_provider_refresh_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn update_provider( &self, _request: tonic::Request, diff --git a/crates/openshell-credential-issuer/src/refresh/engine.rs b/crates/openshell-credential-issuer/src/refresh/engine.rs index d8fef6285c..0fa5bb1a8b 100644 --- a/crates/openshell-credential-issuer/src/refresh/engine.rs +++ b/crates/openshell-credential-issuer/src/refresh/engine.rs @@ -9,7 +9,7 @@ use super::minter::{ }; use super::{ CredentialRefresh, RefreshAttempt, RefreshChange, RefreshOutputs, RefreshRuntime, RefreshState, - RefreshStateStore, minter, + RefreshStateStore, minter, secret_refs, }; use crate::{CredentialIssuer, RefreshMaterialScope, StoreSecrets}; use crate::{Handles, RefreshMaterialDeletion, SecretOwner}; @@ -189,16 +189,21 @@ pub async fn delete_refresh_states_for_provider_with_credentials( } async fn resolve_refresh_material( + store: &Arc, credentials: Option<&Arc>, state: &RefreshState, -) -> Result { +) -> Result { let mut request = minter::MintRequest::from(state); + let bindings = secret_refs::material_secret_bindings(state.metadata.as_ref())?; + if state.secret_material_handles.is_empty() && bindings.is_empty() { + return Ok(request); + } + let credentials = credentials.ok_or_else(|| { + Status::failed_precondition( + "provider refresh material requires the configured credential runtime", + ) + })?; if !state.secret_material_handles.is_empty() { - let credentials = credentials.ok_or_else(|| { - Status::failed_precondition( - "provider refresh material requires the configured credential runtime", - ) - })?; request.material.extend( credentials .resolve_refresh_material( @@ -208,9 +213,79 @@ async fn resolve_refresh_material( .await?, ); } + for (material_key, binding) in &bindings { + let (value, allowed_endpoints) = + resolve_workspace_secret(store, credentials, state.object_workspace(), binding).await?; + request.material.insert(material_key.clone(), value); + request.secret_endpoint_allowlists.push(allowed_endpoints); + } + // Values copied from a workspace secret keep the endpoint constraint the + // secret's owner set. + let constraints = secret_refs::material_endpoint_constraints(state.metadata.as_ref())?; + request.secret_endpoint_allowlists.extend( + constraints + .into_iter() + .filter(|(material_key, _)| request.material.contains_key(material_key)) + .map(|(_, endpoints)| endpoints), + ); Ok(request) } +/// Read a referenced workspace secret's current value. A concurrent update can +/// delete the handle read from the record, so a failed read re-reads the record +/// once before giving up. +async fn resolve_workspace_secret( + store: &Arc, + credentials: &Arc, + workspace: &str, + binding: &secret_refs::MaterialSecretBinding, +) -> Result<(String, Vec), RefreshFailure> { + let mut last_error = None; + for _ in 0..2 { + let Some(material) = store.get_workspace_secret(workspace, &binding.id).await? else { + return Err(RefreshFailure::fix_configuration( + Status::failed_precondition(format!( + "referenced workspace secret '{}' no longer exists; reconfigure the refresh grant", + binding.name + )), + "referenced_secret_missing", + )); + }; + let handles = HashMap::from([( + secret_refs::WORKSPACE_SECRET_VALUE_KEY.to_string(), + material.handle, + )]); + match credentials + .resolve( + SecretOwner::workspace_secret(workspace, &binding.id), + &handles, + ) + .await + { + Ok(resolved) => { + if let Some(value) = resolved + .into_iter() + .find(|secret| secret.credential_key == secret_refs::WORKSPACE_SECRET_VALUE_KEY) + .map(|secret| secret.value) + .filter(|value| !value.is_empty()) + { + return Ok((value, material.allowed_endpoints)); + } + last_error = Some("credential storage returned no value".to_string()); + } + Err(err) => last_error = Some(err.message().to_string()), + } + } + Err(RefreshFailure::retryable( + Status::unavailable(format!( + "referenced workspace secret '{}' could not be read: {}", + binding.name, + last_error.unwrap_or_default() + )), + "referenced_secret_unavailable", + )) +} + pub fn enqueue_pending_secret_deletion( state: &mut RefreshState, material_key: &str, @@ -327,11 +402,14 @@ async fn persist_refresh_failure_state( } pub fn validate_secret_material_references(state: &RefreshState) -> Result<(), Status> { + let bindings = secret_refs::material_secret_bindings(state.metadata.as_ref())?; let mut missing: Vec<_> = state .secret_material_keys .iter() .filter(|key| { - !state.material.contains_key(*key) && !state.secret_material_handles.contains_key(*key) + !state.material.contains_key(*key) + && !state.secret_material_handles.contains_key(*key) + && !bindings.contains_key(*key) }) .cloned() .collect(); @@ -656,9 +734,9 @@ async fn refresh_provider_credential_locked( let expected_version = persist_refresh_state_if_current(store, &state, expected_version) .await? .ok_or_else(|| Status::aborted("provider refresh changed before minting"))?; - let mint_result = match resolve_refresh_material(Some(credentials), &state).await { + let mint_result = match resolve_refresh_material(store, Some(credentials), &state).await { Ok(transient_state) => mint_credential(&transient_state).await, - Err(err) => Err(err.into()), + Err(err) => Err(err), }; match mint_result { Ok(minted) => { @@ -1252,3 +1330,183 @@ impl CredentialRefresh for RefreshRuntime { run_refresh_worker_tick(self).await } } + +#[cfg(test)] +mod secret_reference_tests { + use super::*; + use crate::{EmbeddedCredentialIssuer, IssuerConfig}; + use openshell_core::proto::ObjectMeta; + use std::sync::Mutex; + + /// Serves a scripted sequence of secret records, then repeats the last. + #[derive(Debug)] + struct ScriptedSecrets(Mutex>>); + + #[async_trait] + impl RefreshStateStore for ScriptedSecrets { + async fn get(&self, _: &str, _: &str, _: &str) -> Result, Status> { + Ok(None) + } + async fn get_by_id(&self, _: &str) -> Result, Status> { + Ok(None) + } + async fn list(&self, _: Option<&str>) -> Result, Status> { + Ok(Vec::new()) + } + async fn create(&self, _: &RefreshState) -> Result<(), Status> { + Ok(()) + } + async fn replace(&self, _: &RefreshState, _: u64) -> Result, Status> { + Ok(None) + } + async fn delete(&self, _: &str, _: u64) -> Result<(), Status> { + Ok(()) + } + async fn acquire( + &self, + _: &str, + _: &str, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + async fn get_workspace_secret( + &self, + _: &str, + _: &str, + ) -> Result, Status> { + let mut script = self.0.lock().unwrap(); + let next = if script.len() > 1 { + script.remove(0) + } else { + script[0].clone() + }; + Ok(next.map(|handle| secret_refs::WorkspaceSecretMaterial { + handle, + allowed_endpoints: vec!["https://login.example.com/token".to_string()], + })) + } + } + + async fn store_value(credentials: &Arc, value: &str) -> CredentialHandle { + let values = HashMap::from([( + secret_refs::WORKSPACE_SECRET_VALUE_KEY.to_string(), + value.to_string(), + )]); + let object_id = format!("secret-id-{}", uuid::Uuid::new_v4()); + credentials + .store(StoreSecrets { + owner: SecretOwner::workspace_secret("default", "secret-id"), + object_id: Some(&object_id), + values: &values, + existing_handles: &HashMap::new(), + }) + .await + .unwrap() + .remove(secret_refs::WORKSPACE_SECRET_VALUE_KEY) + .unwrap() + } + + fn binding() -> secret_refs::MaterialSecretBinding { + secret_refs::MaterialSecretBinding { + id: "secret-id".to_string(), + name: "acme".to_string(), + } + } + + fn credentials() -> Arc { + Arc::new( + EmbeddedCredentialIssuer::from_config(&IssuerConfig { + credential_drivers: vec!["test-static".into()], + ..Default::default() + }) + .unwrap(), + ) + } + + #[tokio::test] + async fn a_mint_racing_a_value_update_rereads_the_secret() { + let credentials = credentials(); + let stale = store_value(&credentials, "old-value").await; + let current = store_value(&credentials, "new-value").await; + // The update deleted the value the mint read first. + credentials + .delete( + SecretOwner::workspace_secret("default", "secret-id"), + &HashMap::from([( + secret_refs::WORKSPACE_SECRET_VALUE_KEY.to_string(), + stale.clone(), + )]), + ) + .await + .unwrap(); + let store: Arc = Arc::new(ScriptedSecrets(Mutex::new(vec![ + Some(stale), + Some(current), + ]))); + let (value, endpoints) = + resolve_workspace_secret(&store, &credentials, "default", &binding()) + .await + .unwrap(); + assert_eq!(value, "new-value"); + assert_eq!( + endpoints, + vec!["https://login.example.com/token".to_string()] + ); + } + + #[tokio::test] + async fn an_unreadable_secret_is_retryable_and_a_missing_one_is_not() { + let credentials = credentials(); + let deleted = store_value(&credentials, "value").await; + credentials + .delete( + SecretOwner::workspace_secret("default", "secret-id"), + &HashMap::from([( + secret_refs::WORKSPACE_SECRET_VALUE_KEY.to_string(), + deleted.clone(), + )]), + ) + .await + .unwrap(); + let store: Arc = + Arc::new(ScriptedSecrets(Mutex::new(vec![Some(deleted)]))); + let failure = resolve_workspace_secret(&store, &credentials, "default", &binding()) + .await + .err() + .unwrap(); + assert_eq!(failure.failure_code, "referenced_secret_unavailable"); + assert_eq!(failure.retry_schedule, RefreshRetrySchedule::Short); + + let store: Arc = Arc::new(ScriptedSecrets(Mutex::new(vec![None]))); + let failure = resolve_workspace_secret(&store, &credentials, "default", &binding()) + .await + .err() + .unwrap(); + assert_eq!(failure.failure_code, "referenced_secret_missing"); + assert_eq!(failure.retry_schedule, RefreshRetrySchedule::Configuration); + } + + #[test] + fn a_binding_satisfies_listed_secret_material_but_its_loss_fails_closed() { + let mut metadata = ObjectMeta::default(); + secret_refs::set_material_secret_bindings( + &mut metadata, + &secret_refs::MaterialSecretBindings::from([("client_secret".to_string(), binding())]), + ); + let mut state = RefreshState { + metadata: Some(metadata), + secret_material_keys: vec!["client_secret".to_string()], + ..Default::default() + }; + validate_secret_material_references(&state).unwrap(); + // An older gateway that dropped the binding sees a listed secret key + // with neither a value nor a handle, and refuses to mint. + state.metadata = Some(ObjectMeta::default()); + assert_eq!( + validate_secret_material_references(&state) + .unwrap_err() + .code(), + tonic::Code::FailedPrecondition + ); + } +} diff --git a/crates/openshell-credential-issuer/src/refresh/minter.rs b/crates/openshell-credential-issuer/src/refresh/minter.rs index f1e5dbc2e1..a15bd35ad1 100644 --- a/crates/openshell-credential-issuer/src/refresh/minter.rs +++ b/crates/openshell-credential-issuer/src/refresh/minter.rs @@ -30,6 +30,9 @@ pub struct MintRequest { pub scopes: Vec, pub max_lifetime: Option, pub additional_output_keys: HashMap, + /// Endpoint allowlists of every workspace secret this request carries. The + /// effective token endpoint must appear in each, and redirects are refused. + pub secret_endpoint_allowlists: Vec>, } /// Transient mint output. Intentionally has no Debug or serialization implementation. @@ -365,6 +368,7 @@ async fn mint_oauth2_refresh_token( &form, max_lifetime_seconds(state), OAuthGrantKind::UserRefreshToken, + &state.secret_endpoint_allowlists, ) .await } @@ -390,6 +394,7 @@ async fn mint_oauth2_client_credentials( &form, max_lifetime_seconds(state), OAuthGrantKind::NonInteractive, + &state.secret_endpoint_allowlists, ) .await } @@ -440,11 +445,21 @@ async fn mint_google_service_account_jwt( &form, lifetime_secs, OAuthGrantKind::NonInteractive, + &state.secret_endpoint_allowlists, ) .await } async fn mint_aws_sts_assume_role(state: &MintRequest) -> Result { + // The STS client does not enforce workspace-secret endpoint constraints. + if !state.secret_endpoint_allowlists.is_empty() { + return Err(RefreshFailure::fix_configuration( + Status::failed_precondition( + "aws_sts_assume_role does not support workspace secret material", + ), + "secret_reference_unsupported", + )); + } let role_arn = required_material(&state.material, "role_arn")?; let session_name = material_value(&state.material, &["session_name"]) .unwrap_or_else(|| "openshell-sandbox".to_string()); @@ -568,6 +583,7 @@ async fn request_token( form: &[(String, String)], max_lifetime_seconds: i64, grant_kind: OAuthGrantKind, + endpoint_allowlists: &[Vec], ) -> Result { let parsed = reqwest::Url::parse(token_url) .map_err(|_| Status::invalid_argument("token_url must be an absolute URL"))?; @@ -582,9 +598,19 @@ async fn request_token( } } + // A request carrying a workspace secret may reach only endpoints that every + // referenced secret allows, and never by redirect: a 307 or 308 would resend + // the body to the new location. + let secret_bound = !endpoint_allowlists.is_empty(); + if secret_bound { + ensure_endpoint_allowed(token_url, endpoint_allowlists)?; + } openshell_crypto::tls::ensure_default_provider(); - let client = reqwest::Client::builder() - .timeout(Duration::from_secs(30)) + let mut client = reqwest::Client::builder().timeout(Duration::from_secs(30)); + if secret_bound { + client = client.redirect(reqwest::redirect::Policy::none()); + } + let client = client .build() .map_err(|e| Status::internal(format!("build refresh HTTP client failed: {e}")))?; // GitHub returns form-encoded bodies unless JSON is requested. @@ -625,6 +651,14 @@ async fn request_token( } })?; let status = response.status(); + if secret_bound && status.is_redirection() { + return Err(RefreshFailure::fix_configuration( + Status::failed_precondition( + "token endpoint redirected a request carrying a workspace secret; redirects are not followed", + ), + "secret_endpoint_redirected", + )); + } if !status.is_success() { let body = read_bounded_oauth_error_body(response).await; return Err(classify_oauth_token_error(status, &body, grant_kind)); @@ -843,6 +877,60 @@ pub fn parse_material_i64( .map_err(|_| Status::invalid_argument(format!("{key} material must be a signed integer"))) } +fn ensure_endpoint_allowed( + token_url: &str, + endpoint_allowlists: &[Vec], +) -> Result<(), RefreshFailure> { + let allowed = crate::endpoint::normalize_token_endpoint(token_url).is_ok_and(|normalized| { + endpoint_allowlists + .iter() + .all(|allowlist| allowlist.contains(&normalized)) + }); + if allowed { + Ok(()) + } else { + Err(RefreshFailure::fix_configuration( + Status::failed_precondition( + "token endpoint is not allowed by a referenced workspace secret", + ), + "secret_endpoint_not_allowed", + )) + } +} + +/// The token endpoint a mint for this request will call, for strategies whose +/// request goes to a single OAuth token endpoint. +/// +/// # Errors +/// +/// Returns `FAILED_PRECONDITION` for strategies without such an endpoint, and +/// `INVALID_ARGUMENT` when the endpoint cannot be determined. +pub fn effective_token_endpoint(request: &MintRequest) -> Result { + match ProviderCredentialRefreshStrategy::try_from(request.strategy) + .unwrap_or(ProviderCredentialRefreshStrategy::Unspecified) + { + ProviderCredentialRefreshStrategy::Oauth2RefreshToken + | ProviderCredentialRefreshStrategy::Oauth2ClientCredentials => oauth2_token_url(request), + ProviderCredentialRefreshStrategy::GoogleServiceAccountJwt => Ok(google_token_url(request)), + strategy => Err(Status::failed_precondition(format!( + "refresh strategy '{}' does not support workspace secret material", + refresh_strategy_name(strategy as i32) + ))), + } +} + +/// Material keys a strategy writes back after a successful mint. They belong +/// to one provider and cannot be supplied by a shared workspace secret. +#[must_use] +pub fn strategy_rotated_material_keys( + strategy: ProviderCredentialRefreshStrategy, +) -> &'static [&'static str] { + match strategy { + ProviderCredentialRefreshStrategy::Oauth2RefreshToken => &["refresh_token"], + _ => &[], + } +} + fn oauth2_token_url(state: &MintRequest) -> Result { if let Some(tenant_id) = material_value(&state.material, &["tenant_id"]) { return Ok(format!( @@ -1246,6 +1334,7 @@ mod tests { &[("grant_type".to_string(), "refresh_token".to_string())], 60, OAuthGrantKind::UserRefreshToken, + &[], ) .await .unwrap(); @@ -1290,6 +1379,7 @@ mod tests { .await; let state = MintRequest { + secret_endpoint_allowlists: Vec::new(), additional_output_keys: HashMap::from([ ("secret_access_key".to_string(), "CUSTOM_SECRET".to_string()), ("session_token".to_string(), "CUSTOM_SESSION".to_string()), @@ -1339,6 +1429,7 @@ mod tests { #[tokio::test] async fn aws_sts_mint_rejects_partial_source_credentials() { let state = MintRequest { + secret_endpoint_allowlists: Vec::new(), additional_output_keys: HashMap::from([ ( "secret_access_key".to_string(), @@ -1393,6 +1484,7 @@ mod tests { .await; let state = MintRequest { + secret_endpoint_allowlists: Vec::new(), additional_output_keys: HashMap::from([ ( "secret_access_key".to_string(), @@ -1428,6 +1520,7 @@ mod tests { #[tokio::test] async fn aws_sts_mint_rejects_session_token_without_source_pair() { let state = MintRequest { + secret_endpoint_allowlists: Vec::new(), additional_output_keys: HashMap::from([ ( "secret_access_key".to_string(), @@ -1590,6 +1683,7 @@ mod tests { &[], 60, grant_kind, + &[], ) .await .err() @@ -1678,4 +1772,128 @@ KfNSI8uuA8PLqmUY30I9KFWzN6VDLu00eKa90F4w3CeWRRQWXW1+007tTz3V1mNw 20A24Fi3HGQmXc7NyuLDODTJsWBICuOemCnRkvcxIlxb+ec7jp+XRmzDwKkzSnVN pM2zFU8SeVkvHKlEuoHaP0s= -----END PRIVATE KEY-----"; + + fn secret_bound_refresh_request(token_url: String, allowed: &str) -> MintRequest { + MintRequest { + strategy: ProviderCredentialRefreshStrategy::Oauth2RefreshToken as i32, + material: HashMap::from([ + ("client_id".into(), "client-id".into()), + ("client_secret".into(), "shared-client-secret".into()), + ("refresh_token".into(), "refresh-token".into()), + ]), + token_url, + secret_endpoint_allowlists: vec![vec![ + crate::endpoint::normalize_token_endpoint(allowed).unwrap(), + ]], + ..Default::default() + } + } + + #[tokio::test] + async fn secret_bound_mint_reaches_an_allowed_endpoint() { + let mock_server = MockServer::start().await; + Mock::given(method("POST")) + .and(path("/token")) + .and(body_string_contains("client_secret=shared-client-secret")) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "access_token": "minted-token", + "expires_in": 3600 + }))) + .expect(1) + .mount(&mock_server) + .await; + let token_url = format!("{}/token", mock_server.uri()); + let request = secret_bound_refresh_request(token_url.clone(), &token_url); + let minted = mint_credential(&request).await.unwrap(); + assert_eq!(minted.access_token, "minted-token"); + } + + #[tokio::test] + async fn secret_bound_mint_never_contacts_a_disallowed_endpoint() { + let mock_server = MockServer::start().await; + Mock::given(method("POST")) + .respond_with(ResponseTemplate::new(200)) + .expect(0) + .mount(&mock_server) + .await; + let request = secret_bound_refresh_request( + format!("{}/token", mock_server.uri()), + "https://login.example.com/token", + ); + let failure = mint_credential(&request).await.err().unwrap(); + assert_eq!(failure.failure_code, "secret_endpoint_not_allowed"); + assert_eq!( + failure.recovery_action, + ProviderCredentialRefreshRecoveryAction::FixConfiguration + ); + } + + #[tokio::test] + async fn secret_bound_mint_refuses_redirects_without_contacting_the_target() { + let approved = MockServer::start().await; + let target = MockServer::start().await; + Mock::given(method("POST")) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "access_token": "stolen", + "expires_in": 3600 + }))) + .expect(0) + .mount(&target) + .await; + Mock::given(method("POST")) + .and(path("/token")) + .respond_with( + ResponseTemplate::new(307) + .insert_header("location", format!("{}/token", target.uri()).as_str()), + ) + .expect(1) + .mount(&approved) + .await; + let token_url = format!("{}/token", approved.uri()); + let request = secret_bound_refresh_request(token_url.clone(), &token_url); + let failure = mint_credential(&request).await.err().unwrap(); + assert_eq!(failure.failure_code, "secret_endpoint_redirected"); + assert_eq!( + failure.recovery_action, + ProviderCredentialRefreshRecoveryAction::FixConfiguration + ); + } + + #[tokio::test] + async fn aws_sts_refuses_workspace_secret_material() { + let request = MintRequest { + strategy: ProviderCredentialRefreshStrategy::AwsStsAssumeRole as i32, + secret_endpoint_allowlists: vec![vec!["https://sts.amazonaws.com/".into()]], + ..Default::default() + }; + let failure = mint_credential(&request).await.err().unwrap(); + assert_eq!(failure.failure_code, "secret_reference_unsupported"); + } + + #[test] + fn effective_token_endpoint_covers_oauth_and_google_only() { + let oauth = MintRequest { + strategy: ProviderCredentialRefreshStrategy::Oauth2ClientCredentials as i32, + material: HashMap::from([("tenant_id".into(), "tenant".into())]), + token_url: "https://ignored.example.com/token".into(), + ..Default::default() + }; + assert_eq!( + effective_token_endpoint(&oauth).unwrap(), + "https://login.microsoftonline.com/tenant/oauth2/v2.0/token" + ); + let google = MintRequest { + strategy: ProviderCredentialRefreshStrategy::GoogleServiceAccountJwt as i32, + ..Default::default() + }; + assert_eq!( + effective_token_endpoint(&google).unwrap(), + "https://oauth2.googleapis.com/token" + ); + let aws = MintRequest { + strategy: ProviderCredentialRefreshStrategy::AwsStsAssumeRole as i32, + ..Default::default() + }; + assert!(effective_token_endpoint(&aws).is_err()); + } } diff --git a/crates/openshell-credential-issuer/src/refresh/mod.rs b/crates/openshell-credential-issuer/src/refresh/mod.rs index d93d3265e8..b0b536030d 100644 --- a/crates/openshell-credential-issuer/src/refresh/mod.rs +++ b/crates/openshell-credential-issuer/src/refresh/mod.rs @@ -7,6 +7,7 @@ #[doc(hidden)] pub mod engine; pub mod minter; +pub mod secret_refs; use std::collections::HashMap; use std::sync::Arc; @@ -90,6 +91,7 @@ impl From<&RefreshState> for minter::MintRequest { scopes: state.scopes.clone(), max_lifetime: state.max_lifetime, additional_output_keys: state.additional_output_keys.clone(), + secret_endpoint_allowlists: Vec::new(), } } } @@ -112,6 +114,15 @@ pub trait RefreshStateStore: std::fmt::Debug + Send + Sync { async fn replace(&self, state: &RefreshState, version: u64) -> Result, Status>; async fn delete(&self, id: &str, version: u64) -> Result<(), Status>; async fn acquire(&self, provider_id: &str, key: &str) -> Result, Status>; + /// The current handle and endpoint constraint of a live workspace secret, + /// or `None` when it does not exist or is being deleted. + async fn get_workspace_secret( + &self, + _workspace: &str, + _secret_id: &str, + ) -> Result, Status> { + Ok(None) + } } /// Only staged handles (or the existing inline/test fallback) cross this port. diff --git a/crates/openshell-credential-issuer/src/refresh/secret_refs.rs b/crates/openshell-credential-issuer/src/refresh/secret_refs.rs new file mode 100644 index 0000000000..c5d34083b4 --- /dev/null +++ b/crates/openshell-credential-issuer/src/refresh/secret_refs.rs @@ -0,0 +1,157 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +//! Workspace-secret references and endpoint constraints in refresh state. +//! +//! Both live in refresh-state metadata annotations rather than new protobuf +//! fields: older gateways decode refresh state into a struct that drops unknown +//! fields and rewrite it with compare-and-swap, but they preserve annotations. + +use std::collections::BTreeMap; + +use openshell_core::proto::datamodel::v1::{CredentialHandle, ObjectMeta}; +use serde::de::DeserializeOwned; +use serde::{Deserialize, Serialize}; +use tonic::Status; + +/// Material key to the workspace secret that supplies it. +pub const MATERIAL_SECRETS_ANNOTATION: &str = "openshell.nvidia.com/refresh-material-secrets"; +/// Material key to the token endpoints its value may be sent to. Set when a +/// referenced secret's value is copied into provider-owned material. +pub const MATERIAL_ENDPOINTS_ANNOTATION: &str = "openshell.nvidia.com/refresh-material-endpoints"; +/// Logical key of the single value each workspace secret stores. +pub const WORKSPACE_SECRET_VALUE_KEY: &str = "value"; + +/// A binding from one material key to a workspace secret. The ID is +/// authoritative; the name is recorded for status reads and messages. +#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub struct MaterialSecretBinding { + pub id: String, + pub name: String, +} + +pub type MaterialSecretBindings = BTreeMap; +pub type MaterialEndpointConstraints = BTreeMap>; + +/// The current storage handle and endpoint constraint of a live workspace +/// secret, as the embedding gateway stores them. +pub struct WorkspaceSecretMaterial { + pub handle: CredentialHandle, + pub allowed_endpoints: Vec, +} + +/// Bindings recorded on a refresh state. A malformed annotation fails closed. +/// +/// # Errors +/// +/// Returns `FAILED_PRECONDITION` when the annotation is not valid JSON. +pub fn material_secret_bindings( + metadata: Option<&ObjectMeta>, +) -> Result { + parse_annotation(metadata, MATERIAL_SECRETS_ANNOTATION) +} + +/// Endpoint constraints recorded on a refresh state. A malformed annotation +/// fails closed. +/// +/// # Errors +/// +/// Returns `FAILED_PRECONDITION` when the annotation is not valid JSON. +pub fn material_endpoint_constraints( + metadata: Option<&ObjectMeta>, +) -> Result { + parse_annotation(metadata, MATERIAL_ENDPOINTS_ANNOTATION) +} + +/// Record bindings, removing the annotation when there are none. +pub fn set_material_secret_bindings(metadata: &mut ObjectMeta, bindings: &MaterialSecretBindings) { + set_annotation( + metadata, + MATERIAL_SECRETS_ANNOTATION, + bindings, + bindings.is_empty(), + ); +} + +/// Record endpoint constraints, removing the annotation when there are none. +pub fn set_material_endpoint_constraints( + metadata: &mut ObjectMeta, + constraints: &MaterialEndpointConstraints, +) { + set_annotation( + metadata, + MATERIAL_ENDPOINTS_ANNOTATION, + constraints, + constraints.is_empty(), + ); +} + +fn parse_annotation( + metadata: Option<&ObjectMeta>, + key: &str, +) -> Result { + let Some(raw) = metadata.and_then(|metadata| metadata.annotations.get(key)) else { + return Ok(T::default()); + }; + serde_json::from_str(raw).map_err(|_| { + Status::failed_precondition(format!( + "provider refresh annotation {key} is malformed; reconfigure the refresh grant" + )) + }) +} + +fn set_annotation(metadata: &mut ObjectMeta, key: &str, value: &T, empty: bool) { + if empty { + metadata.annotations.remove(key); + return; + } + // Serializing a map of strings cannot fail. + if let Ok(encoded) = serde_json::to_string(value) { + metadata.annotations.insert(key.to_string(), encoded); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn bindings_round_trip_and_clear() { + let mut metadata = ObjectMeta::default(); + let bindings = MaterialSecretBindings::from([( + "client_secret".to_string(), + MaterialSecretBinding { + id: "secret-id".to_string(), + name: "acme".to_string(), + }, + )]); + set_material_secret_bindings(&mut metadata, &bindings); + assert_eq!(material_secret_bindings(Some(&metadata)).unwrap(), bindings); + + set_material_secret_bindings(&mut metadata, &MaterialSecretBindings::new()); + assert!( + !metadata + .annotations + .contains_key(MATERIAL_SECRETS_ANNOTATION) + ); + assert!( + material_secret_bindings(Some(&metadata)) + .unwrap() + .is_empty() + ); + assert!(material_secret_bindings(None).unwrap().is_empty()); + } + + #[test] + fn malformed_annotations_fail_closed() { + let mut metadata = ObjectMeta::default(); + metadata + .annotations + .insert(MATERIAL_SECRETS_ANNOTATION.to_string(), "{".to_string()); + metadata + .annotations + .insert(MATERIAL_ENDPOINTS_ANNOTATION.to_string(), "[]".to_string()); + assert!(material_secret_bindings(Some(&metadata)).is_err()); + assert!(material_endpoint_constraints(Some(&metadata)).is_err()); + } +} diff --git a/crates/openshell-sdk/tests/client_mock.rs b/crates/openshell-sdk/tests/client_mock.rs index 2fa3f30b22..2898af1503 100644 --- a/crates/openshell-sdk/tests/client_mock.rs +++ b/crates/openshell-sdk/tests/client_mock.rs @@ -892,6 +892,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("unused")) } + async fn inline_provider_refresh_secrets( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + async fn get_sandbox_config( &self, _: tonic::Request, diff --git a/crates/openshell-server/src/auth/audit/catalog.rs b/crates/openshell-server/src/auth/audit/catalog.rs index bcea401fcb..9abdd81117 100644 --- a/crates/openshell-server/src/auth/audit/catalog.rs +++ b/crates/openshell-server/src/auth/audit/catalog.rs @@ -45,6 +45,7 @@ const CATALOG: &[(&str, ActivityId)] = &[ (openshell!("DeleteProviderProfile"), DELETE), (openshell!("ConfigureProviderRefresh"), UPDATE), (openshell!("DeleteProviderRefresh"), DELETE), + (openshell!("InlineProviderRefreshSecrets"), UPDATE), (openshell!("RotateProviderCredential"), UPDATE), (openshell!("CreateSecret"), CREATE), (openshell!("UpdateSecret"), UPDATE), diff --git a/crates/openshell-server/src/grpc/mod.rs b/crates/openshell-server/src/grpc/mod.rs index 6bd29dfebb..e14a54e585 100644 --- a/crates/openshell-server/src/grpc/mod.rs +++ b/crates/openshell-server/src/grpc/mod.rs @@ -75,7 +75,8 @@ use openshell_core::proto::{ }; use openshell_core::proto::{ CreateSecretRequest, DeleteSecretRequest, DeleteSecretResponse, GetSecretRequest, - ListSecretsRequest, ListSecretsResponse, SecretResponse, UpdateSecretRequest, + InlineProviderRefreshSecretsRequest, InlineProviderRefreshSecretsResponse, ListSecretsRequest, + ListSecretsResponse, SecretResponse, UpdateSecretRequest, }; use serde::{Deserialize, Serialize}; use std::collections::BTreeMap; @@ -783,6 +784,13 @@ impl OpenShell for OpenShellService { mutation_replay::run(&self.state, request).await } + async fn inline_provider_refresh_secrets( + &self, + request: Request, + ) -> Result, Status> { + mutation_replay::run(&self.state, request).await + } + async fn delete_provider_refresh( &self, request: Request, @@ -1566,6 +1574,8 @@ pub mod test_support { #[cfg(test)] mod mutation_tests; +#[cfg(test)] +mod secret_refresh_tests; #[cfg(test)] mod tests { diff --git a/crates/openshell-server/src/grpc/mutation_replay/ordinary.rs b/crates/openshell-server/src/grpc/mutation_replay/ordinary.rs index bbdb7feb88..dcd337b820 100644 --- a/crates/openshell-server/src/grpc/mutation_replay/ordinary.rs +++ b/crates/openshell-server/src/grpc/mutation_replay/ordinary.rs @@ -29,7 +29,8 @@ use openshell_core::proto::{ WorkspaceSelector, }; use openshell_core::proto::{ - CreateSecretRequest, DeleteSecretRequest, DeleteSecretResponse, SecretResponse, + CreateSecretRequest, DeleteSecretRequest, DeleteSecretResponse, + InlineProviderRefreshSecretsRequest, InlineProviderRefreshSecretsResponse, SecretResponse, UpdateSecretRequest, }; use openshell_core::{GetResourceVersion, ObjectId}; @@ -935,6 +936,12 @@ refresh_mutation!( "ConfigureProviderRefresh", provider::handle_configure_provider_refresh ); +refresh_mutation!( + InlineProviderRefreshSecretsRequest, + InlineProviderRefreshSecretsResponse, + "InlineProviderRefreshSecrets", + provider::handle_inline_provider_refresh_secrets +); refresh_mutation!( RotateProviderCredentialRequest, RotateProviderCredentialResponse, diff --git a/crates/openshell-server/src/grpc/provider.rs b/crates/openshell-server/src/grpc/provider.rs index 6ef3c3b15d..aad8c606ca 100644 --- a/crates/openshell-server/src/grpc/provider.rs +++ b/crates/openshell-server/src/grpc/provider.rs @@ -34,6 +34,7 @@ use openshell_core::telemetry::{ LifecycleOperation, ProviderProfile as TelemetryProviderProfile, TelemetryOutcome, }; use openshell_credential_issuer::refresh::CredentialRefresh; +use openshell_credential_issuer::refresh::{minter, secret_refs}; use openshell_policy::ProviderPolicyLayer; use prost::Message; use sha2::{Digest, Sha256}; @@ -42,7 +43,7 @@ use tonic::Status; use tracing::warn; use super::validation::{validate_provider_fields, validate_provider_mutable_fields}; -use super::{MAX_MAP_KEY_LEN, MAX_MAP_VALUE_LEN, MAX_PROVIDER_CONFIG_ENTRIES}; +use super::{MAX_MAP_KEY_LEN, MAX_MAP_VALUE_LEN, MAX_NAME_LEN, MAX_PROVIDER_CONFIG_ENTRIES}; const GATEWAY_SPIFFE_WORKLOAD_API_SOCKET: &str = "OPENSHELL_GATEWAY_SPIFFE_WORKLOAD_API_SOCKET"; const EXTERNAL_STABLE_PLACEHOLDER_EPOCH: &str = "external-stable-placeholder-v1"; @@ -2663,6 +2664,7 @@ use openshell_core::proto::{ ExchangeProviderSubjectTokenRequest, ExchangeProviderSubjectTokenResponse, GetProviderProfileRequest, GetProviderRefreshStatusRequest, GetProviderRefreshStatusResponse, GetProviderRequest, ImportProviderProfilesRequest, ImportProviderProfilesResponse, + InlineProviderRefreshSecretsRequest, InlineProviderRefreshSecretsResponse, LintProviderProfilesRequest, LintProviderProfilesResponse, ListProviderProfilesRequest, ListProviderProfilesResponse, ListProvidersRequest, ListProvidersResponse, ProviderProfileDiagnostic, ProviderProfileImportItem, ProviderProfileResponse, @@ -3579,6 +3581,121 @@ fn validate_strategy_profile_binding( Ok(()) } +/// Shape checks for `material_secrets` that need no stored state. +fn validate_material_secret_references( + request: &ConfigureProviderRefreshRequest, +) -> Result<(), Status> { + if request.material_secrets.len() > MAX_PROVIDER_CONFIG_ENTRIES { + return Err(Status::invalid_argument(format!( + "material_secrets exceeds maximum entries ({} > {MAX_PROVIDER_CONFIG_ENTRIES})", + request.material_secrets.len() + ))); + } + for (key, secret) in &request.material_secrets { + if key.is_empty() || key.len() > MAX_MAP_KEY_LEN { + return Err(Status::invalid_argument(format!( + "material_secrets key must be 1 to {MAX_MAP_KEY_LEN} bytes" + ))); + } + if secret.trim().is_empty() || secret.len() > MAX_NAME_LEN { + return Err(Status::invalid_argument(format!( + "material_secrets entry '{key}' must name a secret" + ))); + } + if request.material.contains_key(key) { + return Err(Status::invalid_argument(format!( + "material key '{key}' cannot be supplied by both material and material_secrets" + ))); + } + // Gateways that predate workspace secrets ignore material_secrets but + // reject a listed secret key with no inline value. + if !request + .secret_material_keys + .iter() + .any(|listed| listed == key) + { + return Err(Status::invalid_argument(format!( + "material_secrets key '{key}' must also be listed in secret_material_keys" + ))); + } + } + Ok(()) +} + +/// Only secret minting material may come from a shared workspace secret, and +/// never material the minter writes back for one provider. +fn validate_material_secret_keys( + strategy: ProviderCredentialRefreshStrategy, + refresh_defaults: Option<&CredentialRefreshProfile>, + material_secrets: &HashMap, +) -> Result<(), Status> { + for key in material_secrets.keys() { + if minter::strategy_rotated_material_keys(strategy).contains(&key.as_str()) { + return Err(Status::invalid_argument(format!( + "material key '{key}' is rotated per provider and cannot come from a workspace secret" + ))); + } + let strategy_secret = crate::provider_refresh::strategy_secret_material_keys(strategy) + .contains(&key.as_str()); + let profile_secret = refresh_defaults.is_some_and(|refresh| { + refresh + .material + .iter() + .any(|item| item.secret && item.name == *key) + }); + if !strategy_secret && !profile_secret { + return Err(Status::invalid_argument(format!( + "material key '{key}' is not secret material and cannot come from a workspace secret" + ))); + } + } + Ok(()) +} + +/// Resolve referenced secrets to immutable IDs and check that the token +/// endpoint this refresh will call is one each secret allows. +async fn bind_material_secrets( + state: &ServerState, + workspace: &str, + strategy: ProviderCredentialRefreshStrategy, + material: &HashMap, + token_url: &str, + material_secrets: &HashMap, +) -> Result { + let mut bindings = secret_refs::MaterialSecretBindings::new(); + if material_secrets.is_empty() { + return Ok(bindings); + } + let endpoint = minter::effective_token_endpoint(&minter::MintRequest { + strategy: strategy as i32, + material: material.clone(), + token_url: token_url.to_string(), + ..Default::default() + })?; + let endpoint = openshell_credential_issuer::endpoint::normalize_token_endpoint(&endpoint) + .map_err(|reason| { + Status::failed_precondition(format!("refresh token endpoint {reason}")) + })?; + for (material_key, name) in material_secrets { + let secret = super::secret::get_live_secret(&state.store, workspace, name) + .await? + .ok_or_else(|| Status::not_found(format!("workspace secret '{name}' not found")))?; + if !secret.allowed_endpoints.contains(&endpoint) { + return Err(Status::failed_precondition(format!( + "workspace secret '{name}' does not allow token endpoint {endpoint}" + ))); + } + bindings.insert( + material_key.clone(), + secret_refs::MaterialSecretBinding { + id: secret.object_id().to_string(), + name: name.clone(), + }, + ); + } + Ok(bindings) +} + fn validate_refresh_material( material: &HashMap, refresh_defaults: Option<&CredentialRefreshProfile>, @@ -4999,6 +5116,10 @@ pub(super) async fn handle_configure_provider_refresh( ))); } } + validate_material_secret_references(&request)?; + if !request.material_secrets.is_empty() { + super::secret::ensure_workspace_secrets_active(state).await?; + } if request .material .get("token_url") @@ -5031,7 +5152,10 @@ pub(super) async fn handle_configure_provider_refresh( let has_source_secret_key = request .material .get("aws_secret_access_key") - .is_some_and(|value| !value.trim().is_empty()); + .is_some_and(|value| !value.trim().is_empty()) + || request + .material_secrets + .contains_key("aws_secret_access_key"); if has_source_access_key != has_source_secret_key { return Err(Status::invalid_argument( "aws_access_key_id and aws_secret_access_key must both be set or both omitted", @@ -5112,16 +5236,25 @@ pub(super) async fn handle_configure_provider_refresh( refresh_defaults.as_ref(), &additional_output_keys, )?; - validate_refresh_material(&request.material, refresh_defaults.as_ref())?; + let mut supplied_material = request.material.clone(); + for key in request.material_secrets.keys() { + supplied_material.insert(key.clone(), "workspace-secret".to_string()); + } + validate_refresh_material(&supplied_material, refresh_defaults.as_ref())?; let mut secret_material_keys: HashSet = request.secret_material_keys.iter().cloned().collect(); for key in &request.secret_material_keys { - if !request.material.contains_key(key) { + if !supplied_material.contains_key(key) { return Err(Status::invalid_argument(format!( "secret_material_keys entry '{key}' is not present in material" ))); } } + validate_material_secret_keys( + strategy, + refresh_defaults.as_ref(), + &request.material_secrets, + )?; if let Some(refresh) = refresh_defaults.as_ref() { secret_material_keys.extend( refresh @@ -5144,6 +5277,15 @@ pub(super) async fn handle_configure_provider_refresh( .as_ref() .map(|refresh| refresh.token_url.clone()) .unwrap_or_default(); + let material_secret_bindings = bind_material_secrets( + state, + &workspace, + strategy, + &request.material, + &token_url, + &request.material_secrets, + ) + .await?; let scopes = if material_scopes.is_empty() { refresh_defaults .as_ref() @@ -5256,6 +5398,16 @@ pub(super) async fn handle_configure_provider_refresh( } } crate::provider_refresh::set_refresh_expiration_presence(&mut state_record, has_expiration); + // A reconfiguration replaces every binding and drops endpoint constraints + // on copied values: the caller supplied all material again. + { + let metadata = state_record.metadata.get_or_insert_with(Default::default); + secret_refs::set_material_secret_bindings(metadata, &material_secret_bindings); + secret_refs::set_material_endpoint_constraints( + metadata, + &secret_refs::MaterialEndpointConstraints::new(), + ); + } let material_staging_id = format!( "{}-refresh-config-{}", provider.object_id(), @@ -5391,6 +5543,173 @@ pub(super) async fn handle_rotate_provider_credential( })) } +/// Copy referenced workspace secrets into provider-owned refresh material. +/// +/// Unlike reconfiguration this keeps every other handle, including a refresh +/// token the gateway rotated, and the authorization epoch. Each copied value +/// keeps the endpoint constraint of the secret it came from. Not gated by the +/// workspace-secrets activation, so it works while rolling back. +pub(super) async fn handle_inline_provider_refresh_secrets( + state: &Arc, + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let replay_facts = super::mutation_replay::ordinary::Facts::from_request(&request); + let request = request.into_inner(); + let authz = authorize_workspace( + &state.store, + &state.admin_role, + &principal, + crate::auth::workspace_authz::selected_workspace_name(request.workspace_scope.as_ref())?, + MinWorkspaceRole::Admin, + ) + .await?; + let workspace = super::workspace::resolve_workspace(state.store.as_ref(), &authz.workspace) + .await? + .name; + let provider_name = request.provider.trim(); + let credential_key = request.credential_key.trim(); + if provider_name.is_empty() { + return Err(Status::invalid_argument("provider is required")); + } + if credential_key.is_empty() { + return Err(Status::invalid_argument("credential_key is required")); + } + let _mutation_guard = state + .compute + .mutation_guard(MutationScope::Workspace(&workspace)) + .await + .map_err(|err| super::persistence_error_to_status(err, "acquire provider mutation lock"))?; + let provider = state + .store + .get_message_by_name::(&workspace, provider_name) + .await + .map_err(|e| Status::internal(format!("fetch provider failed: {e}")))? + .ok_or_else(|| Status::not_found("provider not found"))?; + let mut refresh: StoredProviderCredentialRefreshState = state + .refresh + .get(&workspace, provider.object_id(), credential_key) + .await? + .as_ref() + .map(Into::into) + .ok_or_else(|| Status::not_found("provider refresh not found"))?; + if refresh + .metadata + .as_ref() + .is_some_and(|metadata| metadata.deletion_time.is_some()) + { + return Err(Status::failed_precondition( + "provider refresh is being deleted", + )); + } + let bindings = secret_refs::material_secret_bindings(refresh.metadata.as_ref())?; + if bindings.is_empty() { + replay_facts.refresh(&refresh)?; + return Ok(Response::new(InlineProviderRefreshSecretsResponse { + status: Some(crate::provider_refresh::refresh_status_from_state(&refresh)), + })); + } + let mut constraints = secret_refs::material_endpoint_constraints(refresh.metadata.as_ref())?; + let mut values = HashMap::new(); + for (material_key, binding) in &bindings { + let secret = state + .store + .get_message::(&binding.id) + .await + .map_err(|e| Status::internal(format!("fetch workspace secret failed: {e}")))? + .filter(|secret| { + secret.object_workspace() == workspace + && secret + .metadata + .as_ref() + .is_some_and(|metadata| metadata.deletion_time.is_none()) + }); + let Some((secret, handle)) = secret.and_then(|secret| { + let handle = secret.value_handle.clone()?; + Some((secret, handle)) + }) else { + return Err(Status::failed_precondition(format!( + "workspace secret '{}' no longer exists; reconfigure the refresh grant", + binding.name + ))); + }; + let resolved = state + .credentials + .resolve_secrets( + openshell_credential_issuer::SecretOwner::workspace_secret(&workspace, &binding.id), + &HashMap::from([(secret_refs::WORKSPACE_SECRET_VALUE_KEY.to_string(), handle)]), + 0, + ) + .await?; + let value = resolved + .values + .get(secret_refs::WORKSPACE_SECRET_VALUE_KEY) + .cloned() + .ok_or_else(|| { + Status::unavailable(format!( + "workspace secret '{}' could not be read", + binding.name + )) + })?; + values.insert(material_key.clone(), value); + constraints.insert(material_key.clone(), secret.allowed_endpoints); + } + + let staging_id = format!( + "{}-refresh-inline-{}", + provider.object_id(), + uuid::Uuid::new_v4() + ); + let staged = state + .credentials + .store_refresh_material_with_object_id( + crate::provider_refresh::refresh_material_scope(&refresh), + &staging_id, + &values, + &HashMap::new(), + ) + .await?; + refresh.secret_material_handles.extend(staged.clone()); + let expected_version = { + let metadata = refresh.metadata.get_or_insert_with(Default::default); + secret_refs::set_material_secret_bindings( + metadata, + &secret_refs::MaterialSecretBindings::new(), + ); + secret_refs::set_material_endpoint_constraints(metadata, &constraints); + metadata.resource_version + }; + let persisted = state + .refresh + .replace((&refresh).into(), expected_version) + .await; + if !matches!(persisted, Ok(true)) { + if let Err(cleanup_err) = state + .credentials + .delete_refresh_material_handles( + crate::provider_refresh::refresh_material_scope(&refresh), + &staged, + ) + .await + { + warn!( + provider_name = %provider.object_name(), + credential_key, + error = %cleanup_err, + "failed to clean up staged refresh material after inlining failed" + ); + } + return Err(persisted.err().unwrap_or_else(|| { + Status::aborted("provider refresh was concurrently modified; retry") + })); + } + + replay_facts.refresh(&refresh)?; + Ok(Response::new(InlineProviderRefreshSecretsResponse { + status: Some(crate::provider_refresh::refresh_status_from_state(&refresh)), + })) +} + /// Remove the expiry for each key the refresh still owns — one whose stored /// value equals the refresh's `expires_at_ms`. Independently updated expiries (no /// longer matching) are left in place. @@ -8462,6 +8781,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -8555,6 +8875,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -8695,6 +9016,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }; handle_configure_provider_refresh(&state, authed_request(request("original-secret"))) .await @@ -8818,6 +9140,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }; let (first_store_hit, release_first_store) = first_state.credentials.gate_next_store(); @@ -8949,6 +9272,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -9108,6 +9432,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -9178,6 +9503,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -9292,6 +9618,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -9544,6 +9871,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -9627,6 +9955,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -9649,6 +9978,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -9715,6 +10045,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -9735,6 +10066,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -9792,6 +10124,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -12635,6 +12968,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }; handle_configure_provider_refresh(&state, authed_request(configure())) .await @@ -15160,6 +15494,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -15225,6 +15560,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -15302,6 +15638,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -15361,6 +15698,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -15414,6 +15752,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -15495,6 +15834,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -15592,6 +15932,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -15647,6 +15988,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -15833,6 +16175,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }), ) .await @@ -15911,6 +16254,7 @@ mod tests { workspace_scope: Some(openshell_core::proto::workspace_selector( "default".to_string(), )), + material_secrets: HashMap::new(), }) }; diff --git a/crates/openshell-server/src/grpc/secret.rs b/crates/openshell-server/src/grpc/secret.rs index 3418e66b1d..61065165a8 100644 --- a/crates/openshell-server/src/grpc/secret.rs +++ b/crates/openshell-server/src/grpc/secret.rs @@ -18,6 +18,7 @@ use openshell_core::proto::{ }; use openshell_core::{GetResourceVersion, ObjectId, ObjectLabels, ObjectName, ObjectWorkspace}; use openshell_credential_issuer::endpoint::normalize_token_endpoint; +use openshell_credential_issuer::refresh::secret_refs; use openshell_credential_issuer::{Handles, SecretOwner}; use prost::Message; use tonic::{Request, Response, Status}; @@ -29,6 +30,7 @@ use crate::ServerState; use crate::auth::workspace_authz::{ MinWorkspaceRole, authorize_workspace, selected_workspace_name, }; +use crate::compute::MutationScope; use crate::pagination::Pagination; use crate::persistence::{ ObjectListQuery, ObjectType, PersistenceError, Store, WriteCondition, current_time_ms, @@ -36,7 +38,7 @@ use crate::persistence::{ use crate::storage_proto::StoredWorkspaceSecret; /// Logical key of the single value each secret stores. -pub const SECRET_VALUE_KEY: &str = "value"; +pub const SECRET_VALUE_KEY: &str = secret_refs::WORKSPACE_SECRET_VALUE_KEY; const MAX_SECRETS_PER_WORKSPACE: u32 = 500; const MAX_ALLOWED_ENDPOINTS: usize = 16; const MAX_ENDPOINT_LEN: usize = 2048; @@ -536,6 +538,13 @@ pub(super) async fn handle_delete_secret( ) .await? .name; + // Configuring refresh holds this lock too, so no new reference can appear + // between the dependency check and the tombstone. + let _mutation_guard = state + .compute + .mutation_guard(MutationScope::Workspace(&workspace)) + .await + .map_err(|err| super::persistence_error_to_status(err, "acquire secret mutation lock"))?; let stored = state .store .get_message_by_name::(&workspace, &req.name) @@ -546,6 +555,16 @@ pub(super) async fn handle_delete_secret( outcome: super::deletion_outcome(false, req.allow_missing, "secret")?, })); }; + if !is_tombstoned(&stored) { + let dependents = secret_dependents(&state.store, &workspace, stored.object_id()).await?; + if !dependents.is_empty() { + return Err(Status::failed_precondition(format!( + "secret '{}' is referenced by provider refresh configuration: {}", + req.name, + dependents.join(", ") + ))); + } + } let tombstoned = tombstone(state, stored).await?; finish_deletion(state, tombstoned).await?; Ok(Response::new(DeleteSecretResponse { @@ -553,6 +572,36 @@ pub(super) async fn handle_delete_secret( })) } +/// Live refresh configurations in a workspace that reference a secret, named +/// `provider/credential_key`. A malformed binding annotation counts as a +/// reference so deletion fails closed. +async fn secret_dependents( + store: &Store, + workspace: &str, + secret_id: &str, +) -> Result, Status> { + let states = crate::refresh_store::list_refresh_states_in_workspace(store, workspace).await?; + let mut dependents: Vec = states + .iter() + .filter(|state| { + state + .metadata + .as_ref() + .is_some_and(|metadata| metadata.deletion_time.is_none()) + }) + .filter(|state| { + secret_refs::material_secret_bindings(state.metadata.as_ref()) + .map_or(true, |bindings| { + bindings.values().any(|binding| binding.id == secret_id) + }) + }) + .map(|state| format!("{}/{}", state.provider_name, state.credential_key)) + .collect(); + dependents.sort(); + dependents.dedup(); + Ok(dependents) +} + /// Mark a secret deleted and queue its value handle for backend deletion. A /// tombstoned secret is invisible to reads and cannot be referenced. async fn tombstone( diff --git a/crates/openshell-server/src/grpc/secret_refresh_tests.rs b/crates/openshell-server/src/grpc/secret_refresh_tests.rs new file mode 100644 index 0000000000..40d5d824d7 --- /dev/null +++ b/crates/openshell-server/src/grpc/secret_refresh_tests.rs @@ -0,0 +1,509 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +//! Provider refresh configured with workspace-secret material. + +use std::collections::HashMap; +use std::sync::Arc; + +use openshell_core::proto::datamodel::v1::ObjectMeta; +use openshell_core::proto::{ + ConfigureProviderRefreshRequest, CreateSecretRequest, DeleteSecretRequest, + GetProviderRefreshStatusRequest, ImportProviderProfilesRequest, + InlineProviderRefreshSecretsRequest, Provider, ProviderCredentialRefresh, + ProviderCredentialRefreshMaterial, ProviderCredentialRefreshStatus, + ProviderCredentialRefreshStrategy, ProviderProfile, ProviderProfileCategory, + ProviderProfileCredential, ProviderProfileImportItem, RotateProviderCredentialRequest, + UpdateSecretRequest, workspace_selector, +}; +use openshell_core::{ObjectId, ObjectWorkspace}; +use openshell_credential_issuer::refresh::{CredentialRefresh, secret_refs}; +use tonic::Code; +use wiremock::matchers::{body_string_contains, method, path}; +use wiremock::{Mock, MockServer, ResponseTemplate}; + +use super::provider::{ + create_provider_record, handle_configure_provider_refresh, handle_get_provider_refresh_status, + handle_import_provider_profiles, handle_inline_provider_refresh_secrets, + handle_rotate_provider_credential, +}; +use super::secret::{handle_create_secret, handle_delete_secret, handle_update_secret}; +use super::test_support::{authed_request, test_server_state}; +use crate::ServerState; + +const PROFILE: &str = "shared-oauth"; +const PROVIDER: &str = "alice-mcp"; +const KEY: &str = "MCP_ACCESS_TOKEN"; + +fn default_workspace() -> openshell_core::proto::WorkspaceSelector { + workspace_selector("default".to_string()) +} + +async fn enabled_state() -> Arc { + let mut state = test_server_state().await; + Arc::get_mut(&mut state) + .expect("test state is not shared yet") + .config + .enable_workspace_secrets = true; + state +} + +fn material(name: &str, secret: bool) -> ProviderCredentialRefreshMaterial { + ProviderCredentialRefreshMaterial { + name: name.to_string(), + description: String::new(), + required: secret, + secret, + } +} + +async fn import_profile( + state: &Arc, + strategy: ProviderCredentialRefreshStrategy, + token_url: &str, +) { + let credential = ProviderProfileCredential { + token_grant_owners: Vec::new(), + name: "access_token".to_string(), + description: String::new(), + env_vars: vec![KEY.to_string()], + required: true, + stable_placeholder: false, + auth_style: "bearer".to_string(), + header_name: "authorization".to_string(), + query_param: String::new(), + path_template: String::new(), + refresh: Some(ProviderCredentialRefresh { + strategy: strategy as i32, + token_url: token_url.to_string(), + scopes: Vec::new(), + refresh_before: Some(prost_types::Duration { + seconds: 300, + nanos: 0, + }), + max_lifetime: Some(prost_types::Duration { + seconds: 3600, + nanos: 0, + }), + additional_outputs: Vec::new(), + material: vec![ + material("client_id", false), + material("client_secret", true), + ], + }), + token_grant: None, + }; + let profile = ProviderProfile { + id: PROFILE.to_string(), + display_name: "Shared OAuth".to_string(), + category: ProviderProfileCategory::Other as i32, + credentials: vec![credential], + ..Default::default() + }; + handle_import_provider_profiles( + state, + authed_request(ImportProviderProfilesRequest { + request_id: String::new(), + profiles: vec![ProviderProfileImportItem { + profile: Some(profile), + source: format!("{PROFILE}.yaml"), + }], + workspace_scope: Some(default_workspace()), + }), + ) + .await + .expect("import profile"); + create_provider_record( + state.store.as_ref(), + "default", + Provider { + metadata: Some(ObjectMeta { + name: PROVIDER.to_string(), + workspace: "default".to_string(), + ..Default::default() + }), + r#type: PROFILE.to_string(), + credentials: HashMap::from([(KEY.to_string(), "initial-token".to_string())]), + profile_workspace: "default".to_string(), + ..Default::default() + }, + ) + .await + .expect("create provider"); +} + +async fn create_secret(state: &Arc, value: &str, allowed_endpoint: &str) { + handle_create_secret( + state, + authed_request(CreateSecretRequest { + workspace_scope: Some(default_workspace()), + name: "acme".to_string(), + description: String::new(), + value: value.to_string(), + allowed_endpoints: vec![allowed_endpoint.to_string()], + labels: HashMap::new(), + request_id: String::new(), + }), + ) + .await + .expect("create secret"); +} + +fn configure_request( + strategy: ProviderCredentialRefreshStrategy, +) -> ConfigureProviderRefreshRequest { + ConfigureProviderRefreshRequest { + workspace_scope: Some(default_workspace()), + provider: PROVIDER.to_string(), + credential_key: KEY.to_string(), + strategy: strategy as i32, + material: HashMap::from([("client_id".to_string(), "client-id".to_string())]), + secret_material_keys: vec!["client_secret".to_string()], + material_secrets: HashMap::from([("client_secret".to_string(), "acme".to_string())]), + ..Default::default() + } +} + +async fn configure( + state: &Arc, + request: ConfigureProviderRefreshRequest, +) -> Result { + handle_configure_provider_refresh(state, authed_request(request)) + .await + .map(|response| response.into_inner().status.expect("status")) +} + +async fn rotate(state: &Arc) -> Result<(), tonic::Status> { + handle_rotate_provider_credential( + state, + authed_request(RotateProviderCredentialRequest { + workspace_scope: Some(default_workspace()), + provider: PROVIDER.to_string(), + credential_key: KEY.to_string(), + request_id: String::new(), + }), + ) + .await + .map(|_| ()) +} + +async fn delete_secret(state: &Arc) -> Result<(), tonic::Status> { + handle_delete_secret( + state, + authed_request(DeleteSecretRequest { + workspace_scope: Some(default_workspace()), + name: "acme".to_string(), + allow_missing: false, + request_id: String::new(), + }), + ) + .await + .map(|_| ()) +} + +async fn refresh_state( + state: &Arc, +) -> openshell_credential_issuer::refresh::RefreshState { + let provider = state + .store + .get_message_by_name::("default", PROVIDER) + .await + .unwrap() + .unwrap(); + state + .refresh + .get("default", provider.object_id(), KEY) + .await + .unwrap() + .expect("refresh state") +} + +async fn mount_token(server: &MockServer, client_secret: &str, access_token: &str) { + Mock::given(method("POST")) + .and(path("/token")) + .and(body_string_contains(format!( + "client_secret={client_secret}" + ))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "access_token": access_token, + "expires_in": 3600 + }))) + .up_to_n_times(1) + .expect(1) + .mount(server) + .await; +} + +#[tokio::test] +async fn refresh_uses_the_current_secret_value_and_survives_inlining() { + let server = MockServer::start().await; + let token_url = format!("{}/token", server.uri()); + mount_token(&server, "first-secret", "token-1").await; + mount_token(&server, "second-secret", "token-2").await; + + let state = enabled_state().await; + import_profile( + &state, + ProviderCredentialRefreshStrategy::Oauth2ClientCredentials, + &token_url, + ) + .await; + create_secret(&state, "first-secret", &token_url).await; + + let status = configure( + &state, + configure_request(ProviderCredentialRefreshStrategy::Oauth2ClientCredentials), + ) + .await + .expect("configure with a workspace secret"); + assert_eq!( + status.material_secrets, + HashMap::from([("client_secret".to_string(), "acme".to_string())]) + ); + let stored = refresh_state(&state).await; + assert!( + stored + .secret_material_keys + .contains(&"client_secret".to_string()) + ); + assert!(!stored.secret_material_handles.contains_key("client_secret")); + assert!(!stored.material.contains_key("client_secret")); + + rotate(&state).await.expect("mint with the first value"); + + // Rotating the shared secret reaches the provider's next mint. + handle_update_secret( + &state, + authed_request(UpdateSecretRequest { + workspace_scope: Some(default_workspace()), + name: "acme".to_string(), + value: Some("second-secret".to_string()), + ..Default::default() + }), + ) + .await + .expect("rotate the shared secret"); + rotate(&state).await.expect("mint with the second value"); + + let err = delete_secret(&state).await.unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + assert!( + err.message().contains(&format!("{PROVIDER}/{KEY}")), + "{}", + err.message() + ); + + let inlined = handle_inline_provider_refresh_secrets( + &state, + authed_request(InlineProviderRefreshSecretsRequest { + workspace_scope: Some(default_workspace()), + provider: PROVIDER.to_string(), + credential_key: KEY.to_string(), + request_id: String::new(), + }), + ) + .await + .expect("inline secrets") + .into_inner() + .status + .unwrap(); + assert!(inlined.material_secrets.is_empty()); + let stored = refresh_state(&state).await; + assert!(stored.secret_material_handles.contains_key("client_secret")); + let constraints = secret_refs::material_endpoint_constraints(stored.metadata.as_ref()).unwrap(); + assert_eq!( + constraints.get("client_secret"), + Some(&vec![ + openshell_credential_issuer::endpoint::normalize_token_endpoint(&token_url).unwrap() + ]) + ); + + // The secret is no longer referenced, and the copied value still mints. + delete_secret(&state) + .await + .expect("delete unreferenced secret"); + Mock::given(method("POST")) + .and(path("/token")) + .and(body_string_contains("client_secret=second-secret")) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "access_token": "token-3", + "expires_in": 3600 + }))) + .expect(1) + .mount(&server) + .await; + rotate(&state).await.expect("mint with the inlined value"); + + let status = handle_get_provider_refresh_status( + &state, + authed_request(GetProviderRefreshStatusRequest { + workspace_scope: Some(default_workspace()), + provider: PROVIDER.to_string(), + credential_key: KEY.to_string(), + }), + ) + .await + .unwrap() + .into_inner(); + assert!(status.credentials[0].material_secrets.is_empty()); +} + +#[tokio::test] +async fn configure_rejects_unsafe_secret_references() { + let state = enabled_state().await; + let token_url = "https://login.example.com/token"; + import_profile( + &state, + ProviderCredentialRefreshStrategy::Oauth2ClientCredentials, + token_url, + ) + .await; + create_secret(&state, "value", "https://other.example.com/token").await; + let strategy = ProviderCredentialRefreshStrategy::Oauth2ClientCredentials; + + let mut unlisted = configure_request(strategy); + unlisted.secret_material_keys.clear(); + let err = configure(&state, unlisted).await.unwrap_err(); + assert_eq!(err.code(), Code::InvalidArgument); + assert!(err.message().contains("secret_material_keys")); + + let mut both = configure_request(strategy); + both.material + .insert("client_secret".to_string(), "inline".to_string()); + assert_eq!( + configure(&state, both).await.unwrap_err().code(), + Code::InvalidArgument + ); + + let mut not_secret = configure_request(strategy); + not_secret.material.remove("client_id"); + not_secret + .material_secrets + .insert("client_id".to_string(), "acme".to_string()); + not_secret + .secret_material_keys + .push("client_id".to_string()); + let err = configure(&state, not_secret).await.unwrap_err(); + assert_eq!(err.code(), Code::InvalidArgument); + assert!(err.message().contains("not secret material")); + + let mut missing = configure_request(strategy); + missing + .material_secrets + .insert("client_secret".to_string(), "absent".to_string()); + assert_eq!( + configure(&state, missing).await.unwrap_err().code(), + Code::NotFound + ); + + // The profile's token endpoint is not one the secret allows. + let err = configure(&state, configure_request(strategy)) + .await + .unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + assert!(err.message().contains("does not allow token endpoint")); + + // A Microsoft tenant override changes the endpoint the mint would call. + let mut tenant = configure_request(strategy); + tenant + .material + .insert("tenant_id".to_string(), "attacker".to_string()); + assert_eq!( + configure(&state, tenant).await.unwrap_err().code(), + Code::FailedPrecondition + ); +} + +#[tokio::test] +async fn rotating_refresh_tokens_stay_provider_owned() { + let state = enabled_state().await; + let token_url = "https://login.example.com/token"; + import_profile( + &state, + ProviderCredentialRefreshStrategy::Oauth2RefreshToken, + token_url, + ) + .await; + create_secret(&state, "value", token_url).await; + let mut request = configure_request(ProviderCredentialRefreshStrategy::Oauth2RefreshToken); + request + .material_secrets + .insert("refresh_token".to_string(), "acme".to_string()); + request + .secret_material_keys + .push("refresh_token".to_string()); + let err = configure(&state, request).await.unwrap_err(); + assert_eq!(err.code(), Code::InvalidArgument); + assert!(err.message().contains("rotated per provider")); +} + +#[tokio::test] +async fn references_require_activation() { + let state = enabled_state().await; + let token_url = "https://login.example.com/token"; + import_profile( + &state, + ProviderCredentialRefreshStrategy::Oauth2ClientCredentials, + token_url, + ) + .await; + create_secret(&state, "value", token_url).await; + let mut state = state; + Arc::get_mut(&mut state) + .expect("test state is not shared") + .config + .enable_workspace_secrets = false; + let err = configure( + &state, + configure_request(ProviderCredentialRefreshStrategy::Oauth2ClientCredentials), + ) + .await + .unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + assert!(err.message().contains("enable_workspace_secrets")); +} + +#[tokio::test] +async fn a_refresh_whose_secret_disappeared_fails_closed() { + let server = MockServer::start().await; + Mock::given(method("POST")) + .respond_with(ResponseTemplate::new(200)) + .expect(0) + .mount(&server) + .await; + let token_url = format!("{}/token", server.uri()); + let state = enabled_state().await; + import_profile( + &state, + ProviderCredentialRefreshStrategy::Oauth2ClientCredentials, + &token_url, + ) + .await; + create_secret(&state, "value", &token_url).await; + configure( + &state, + configure_request(ProviderCredentialRefreshStrategy::Oauth2ClientCredentials), + ) + .await + .unwrap(); + + // Simulate a secret removed behind the delete guard. + let secret = super::secret::get_live_secret(&state.store, "default", "acme") + .await + .unwrap() + .unwrap(); + state + .store + .delete( + ::object_type(), + secret.object_id(), + ) + .await + .unwrap(); + assert_eq!(secret.object_workspace(), "default"); + + let err = rotate(&state).await.unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + let stored = refresh_state(&state).await; + assert_eq!(stored.failure_code, "referenced_secret_missing"); +} diff --git a/crates/openshell-server/src/provider_refresh.rs b/crates/openshell-server/src/provider_refresh.rs index 4c7a6c5970..110a32327e 100644 --- a/crates/openshell-server/src/provider_refresh.rs +++ b/crates/openshell-server/src/provider_refresh.rs @@ -121,6 +121,16 @@ pub fn refresh_status_from_state( ) .ok() .flatten(), + // A malformed annotation is reported by the refresh itself; status + // reads stay available. + material_secrets: + openshell_credential_issuer::refresh::secret_refs::material_secret_bindings( + state.metadata.as_ref(), + ) + .unwrap_or_default() + .into_iter() + .map(|(material_key, binding)| (material_key, binding.name)) + .collect(), } } diff --git a/crates/openshell-server/src/refresh_store.rs b/crates/openshell-server/src/refresh_store.rs index 2748d16181..1b9f14ba21 100644 --- a/crates/openshell-server/src/refresh_store.rs +++ b/crates/openshell-server/src/refresh_store.rs @@ -5,11 +5,12 @@ use crate::persistence::{ObjectListQuery, ObjectType, PersistenceError, Store, WriteCondition}; use crate::storage_proto::{ StoredProviderCredentialRefreshStateV2 as StoredProviderCredentialRefreshState, - StoredRefreshMaterialDeletion, + StoredRefreshMaterialDeletion, StoredWorkspaceSecret, }; use async_trait::async_trait; use openshell_core::{ObjectId, ObjectName, ObjectWorkspace}; use openshell_credential_issuer::RefreshMaterialDeletion; +use openshell_credential_issuer::refresh::secret_refs::WorkspaceSecretMaterial; use openshell_credential_issuer::refresh::{ RefreshLease, RefreshState, RefreshStateStore, refresh_state_name, }; @@ -120,6 +121,16 @@ pub async fn list_all_refresh_states( .map_err(|e| Status::internal(format!("list provider refresh states failed: {e}"))) } +pub async fn list_refresh_states_in_workspace( + store: &Store, + workspace: &str, +) -> Result, Status> { + store + .collect_messages(ObjectListQuery::Workspace(workspace)) + .await + .map_err(|e| Status::internal(format!("list provider refresh states failed: {e}"))) +} + pub async fn get_refresh_state( store: &Store, workspace: &str, @@ -271,6 +282,31 @@ impl RefreshStateStore for ServerRefreshStore { .map(|guard| -> Box { Box::new(guard) }) .map_err(|_| Status::unavailable("credential refresh coordination unavailable")) } + async fn get_workspace_secret( + &self, + workspace: &str, + secret_id: &str, + ) -> Result, Status> { + let stored = self + .0 + .get_message::(secret_id) + .await + .map_err(|e| Status::internal(format!("fetch workspace secret failed: {e}")))?; + Ok(stored + .filter(|stored| { + stored.object_workspace() == workspace + && stored + .metadata + .as_ref() + .is_some_and(|metadata| metadata.deletion_time.is_none()) + }) + .and_then(|stored| { + stored.value_handle.map(|handle| WorkspaceSecretMaterial { + handle, + allowed_endpoints: stored.allowed_endpoints, + }) + })) + } } #[cfg(test)] diff --git a/crates/openshell-server/src/storage_proto.rs b/crates/openshell-server/src/storage_proto.rs index 64a5cb03b7..116af51f45 100644 --- a/crates/openshell-server/src/storage_proto.rs +++ b/crates/openshell-server/src/storage_proto.rs @@ -226,7 +226,7 @@ mod tests { // updates use a separate capability/service and add one classified RPC. // Discovery RPCs move to ServiceDiscovery; message and durable closures are unchanged. const PUBLIC_RPC_SCHEMA_SHA256: &str = - "76d166f76b8aa0abc7d7611b642e5f499b762c746b37f7f183e401cb0ea23685"; + "2cd18038b7655d784ad2819eabc1d602d2690f887378282921872277d1a91dc6"; const DURABLE_SCHEMA_SHA256: &str = "b6a1de55ca3bdf2ccbf88a32f57e87db84b775c48fcd5f9fcf7811eae8d67c70"; const PUBLIC_DURABLE_OVERLAP_SHA256: &str = @@ -703,7 +703,7 @@ mod tests { } assert_eq!( compiled_method_count, - 118 + PROVIDER_READINESS_RPC_SIGNATURES.len() + 119 + PROVIDER_READINESS_RPC_SIGNATURES.len() + PEER_OWNER_RPC_SIGNATURES.len() + PEER_CONFIG_RPC_SIGNATURES.len() + isolation_methods.len() @@ -713,7 +713,7 @@ mod tests { ); assert_eq!( methods.len(), - 88 + PROVIDER_READINESS_RPC_SIGNATURES.len() + 89 + PROVIDER_READINESS_RPC_SIGNATURES.len() + PEER_OWNER_RPC_SIGNATURES.len() + PEER_CONFIG_RPC_SIGNATURES.len(), "inventory every public gateway RPC" @@ -723,7 +723,7 @@ mod tests { .iter() .filter(|method| method.starts_with("openshell.v1.OpenShell/")) .count(), - 85 + PROVIDER_READINESS_RPC_SIGNATURES.len() + 86 + PROVIDER_READINESS_RPC_SIGNATURES.len() + PEER_OWNER_RPC_SIGNATURES.len() + PEER_CONFIG_RPC_SIGNATURES.len() ); @@ -774,7 +774,7 @@ mod tests { overlap_hash.as_str(), ), ( - (356, 31), + (360, 31), (101, 22), (86, 22), PUBLIC_RPC_SCHEMA_SHA256, diff --git a/crates/openshell-server/tests/common/mod.rs b/crates/openshell-server/tests/common/mod.rs index 13628249dd..ff40ab1ed4 100644 --- a/crates/openshell-server/tests/common/mod.rs +++ b/crates/openshell-server/tests/common/mod.rs @@ -449,6 +449,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn inline_provider_refresh_secrets( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn update_provider( &self, _request: tonic::Request, diff --git a/crates/openshell-server/tests/supervisor_relay_integration.rs b/crates/openshell-server/tests/supervisor_relay_integration.rs index 1e6cc0c96d..8e20e12887 100644 --- a/crates/openshell-server/tests/supervisor_relay_integration.rs +++ b/crates/openshell-server/tests/supervisor_relay_integration.rs @@ -476,6 +476,13 @@ impl OpenShell for RelayGateway { ) -> Result, Status> { Err(Status::unimplemented("unused")) } + + async fn inline_provider_refresh_secrets( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } async fn get_provider_refresh_status( &self, _: tonic::Request, diff --git a/proto/openshell.proto b/proto/openshell.proto index fb5fe46994..7cc3ef68f3 100644 --- a/proto/openshell.proto +++ b/proto/openshell.proto @@ -421,6 +421,18 @@ service OpenShell { }; } + // Copy each workspace secret referenced by one credential's refresh + // configuration into provider-owned refresh material and remove the + // references. Other material, including a rotated refresh token, is kept. + rpc InlineProviderRefreshSecrets(InlineProviderRefreshSecretsRequest) + returns (InlineProviderRefreshSecretsResponse) { + option (openshell.options.v1.authorization) = { + auth_mode: "bearer" + scope: "provider:write" + workspace_role: "admin" + }; + } + // Delete gateway-owned refresh configuration for one provider credential. rpc DeleteProviderRefresh(DeleteProviderRefreshRequest) returns (DeleteProviderRefreshResponse) { @@ -2608,6 +2620,8 @@ message ProviderCredentialRefreshStatus { // values are not persisted or returned. string provider_error_subtype = 12; google.protobuf.Timestamp last_error_time = 113; + // Material key to the name of the workspace secret that supplies it. + map material_secrets = 14; } // Provider profile local discovery declaration. @@ -2637,13 +2651,19 @@ message ConfigureProviderRefreshRequest { ProviderCredentialRefreshStrategy strategy = 3; map material = 4 [(openshell.options.v1.secret) = true]; // Additional material names the caller requests be stored as secrets. Every - // name must be present in material. The server also classifies secrets from - // the authoritative provider profile and refresh strategy. + // name must be present in material or material_secrets. The server also + // classifies secrets from the authoritative provider profile and refresh + // strategy. repeated string secret_material_keys = 5; google.protobuf.Timestamp expiration_time = 106; // Optional nonzero UUID for durable at-most-once admission. Successful results // can be replayed for 24 hours; see the API errors and retries reference. string request_id = 8; + // Secret material supplied by a workspace secret in the same workspace: each + // key is a material key and each value names a Secret. Every key must also be + // listed in secret_material_keys, so gateways that predate workspace secrets + // reject the request instead of ignoring this field. + map material_secrets = 9; } message ConfigureProviderRefreshResponse { @@ -2664,6 +2684,20 @@ message RotateProviderCredentialResponse { ProviderCredentialRefreshStatus status = 1; } +message InlineProviderRefreshSecretsRequest { + // Workspace scope. Only a named workspace selection is accepted. + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string provider = 2; + string credential_key = 3; + // Optional nonzero UUID for durable at-most-once admission. Successful results + // can be replayed for 24 hours; see the API errors and retries reference. + string request_id = 4; +} + +message InlineProviderRefreshSecretsResponse { + ProviderCredentialRefreshStatus status = 1; +} + message DeleteProviderRefreshRequest { // Workspace scope. Only a named workspace selection is accepted. openshell.datamodel.v1.WorkspaceSelector workspace_scope = 4; From 9ecdfe6c9e48f73767642075326244ebee6b3778 Mon Sep 17 00:00:00 2001 From: Mrunal Patel Date: Wed, 18 Nov 2026 08:14:46 +0000 Subject: [PATCH 04/10] feat(server): add RevealSecret for control-plane callers Return a workspace secret's value to operators, or to OIDC workspace admins whose token carries the literal secret:reveal scope on a gateway that enforces scopes. openshell:all, unscoped OIDC gateways, user mTLS certificates, and local development identities are rejected in the handler, independent of middleware scope handling. The RPC is disabled unless enable_secret_reveal is set, and every call is audited. Group the workspace secret settings under WorkspaceSecretsConfig. Refs #5563 Signed-off-by: Mrunal Patel --- .../tests/ensure_providers_integration.rs | 7 + .../openshell-cli/tests/mtls_integration.rs | 7 + .../tests/provider_commands_integration.rs | 7 + .../sandbox_create_lifecycle_integration.rs | 7 + .../sandbox_exec_streaming_integration.rs | 1 + .../sandbox_name_fallback_integration.rs | 7 + crates/openshell-core/src/config.rs | 21 +- crates/openshell-sdk/tests/client_mock.rs | 7 + .../src/auth/audit/catalog.rs | 13 +- crates/openshell-server/src/cli.rs | 12 +- crates/openshell-server/src/config_file.rs | 3 + crates/openshell-server/src/grpc/mod.rs | 17 +- .../src/grpc/mutation_audit.rs | 4 + crates/openshell-server/src/grpc/secret.rs | 206 +++++++++++++++++- .../src/grpc/secret_refresh_tests.rs | 6 +- crates/openshell-server/src/storage_proto.rs | 10 +- crates/openshell-server/tests/common/mod.rs | 7 + .../tests/supervisor_relay_integration.rs | 7 + proto/openshell.proto | 25 +++ 19 files changed, 353 insertions(+), 21 deletions(-) diff --git a/crates/openshell-cli/tests/ensure_providers_integration.rs b/crates/openshell-cli/tests/ensure_providers_integration.rs index dd4dc61a80..8d5145ece3 100644 --- a/crates/openshell-cli/tests/ensure_providers_integration.rs +++ b/crates/openshell-cli/tests/ensure_providers_integration.rs @@ -503,6 +503,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn reveal_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn inline_provider_refresh_secrets( &self, _request: tonic::Request, diff --git a/crates/openshell-cli/tests/mtls_integration.rs b/crates/openshell-cli/tests/mtls_integration.rs index 10e38b2356..59a3761698 100644 --- a/crates/openshell-cli/tests/mtls_integration.rs +++ b/crates/openshell-cli/tests/mtls_integration.rs @@ -418,6 +418,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn reveal_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn inline_provider_refresh_secrets( &self, _request: tonic::Request, diff --git a/crates/openshell-cli/tests/provider_commands_integration.rs b/crates/openshell-cli/tests/provider_commands_integration.rs index f0d48676d4..14c1eb9137 100644 --- a/crates/openshell-cli/tests/provider_commands_integration.rs +++ b/crates/openshell-cli/tests/provider_commands_integration.rs @@ -1455,6 +1455,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn reveal_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn inline_provider_refresh_secrets( &self, _request: tonic::Request, diff --git a/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs b/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs index 0fbe659e43..d109d21193 100644 --- a/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs +++ b/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs @@ -706,6 +706,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn reveal_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn inline_provider_refresh_secrets( &self, _request: tonic::Request, diff --git a/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs b/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs index 44f93d672a..841e492365 100644 --- a/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs +++ b/crates/openshell-cli/tests/sandbox_exec_streaming_integration.rs @@ -266,6 +266,7 @@ mock_gateway! { list_secrets(proto::ListSecretsRequest) -> proto::ListSecretsResponse; update_secret(proto::UpdateSecretRequest) -> proto::SecretResponse; delete_secret(proto::DeleteSecretRequest) -> proto::DeleteSecretResponse; + reveal_secret(proto::RevealSecretRequest) -> proto::RevealSecretResponse; inline_provider_refresh_secrets(proto::InlineProviderRefreshSecretsRequest) -> proto::InlineProviderRefreshSecretsResponse; get_sandbox_config(proto::GetSandboxConfigRequest) -> proto::GetSandboxConfigResponse; get_gateway_config(proto::GetGatewayConfigRequest) -> proto::GetGatewayConfigResponse; diff --git a/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs b/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs index 55598ca08e..90b52e3be4 100644 --- a/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs +++ b/crates/openshell-cli/tests/sandbox_name_fallback_integration.rs @@ -476,6 +476,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn reveal_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn inline_provider_refresh_secrets( &self, _request: tonic::Request, diff --git a/crates/openshell-core/src/config.rs b/crates/openshell-core/src/config.rs index 96ae0274c2..8571dcf545 100644 --- a/crates/openshell-core/src/config.rs +++ b/crates/openshell-core/src/config.rs @@ -309,10 +309,8 @@ pub struct Config { /// Disabled by default; not a license to run mixed-version writers. pub enable_sandbox_updates: bool, - /// Operator activation for creating workspace secrets and referencing them - /// from provider refresh configuration. Enable only after every gateway - /// that can serve traffic supports workspace secrets. Disabled by default. - pub enable_workspace_secrets: bool, + /// Workspace secret activation and reveal settings. + pub workspace_secrets: WorkspaceSecretsConfig, /// Disabled-by-default gateway interceptor service configs. pub gateway_interceptors: Vec, @@ -570,6 +568,19 @@ pub struct MetricsTlsConfig { pub require_client_auth: bool, } +/// Workspace secret settings. Both default to off. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)] +pub struct WorkspaceSecretsConfig { + /// Operator activation for creating workspace secrets and referencing them + /// from provider refresh configuration. Enable only after every gateway + /// that can serve traffic supports workspace secrets. + pub enabled: bool, + /// Allow `RevealSecret` to return values to operators and to OIDC tokens + /// holding the literal `secret:reveal` scope. Enable only for a + /// control-plane service that needs it. + pub reveal_enabled: bool, +} + /// OIDC (`OpenID` Connect) configuration for JWT-based authentication. /// /// When configured, the server validates `authorization: Bearer ` @@ -1116,7 +1127,7 @@ impl Config { auth: GatewayAuthConfig::default(), enable_websocket_tunnel: false, enable_sandbox_updates: false, - enable_workspace_secrets: false, + workspace_secrets: WorkspaceSecretsConfig::default(), gateway_interceptors: Vec::new(), provider_profile_sources: vec![GatewayProviderProfileSourceConfig::User], mtls_auth: MtlsAuthConfig::default(), diff --git a/crates/openshell-sdk/tests/client_mock.rs b/crates/openshell-sdk/tests/client_mock.rs index 2898af1503..4bd8a545bc 100644 --- a/crates/openshell-sdk/tests/client_mock.rs +++ b/crates/openshell-sdk/tests/client_mock.rs @@ -892,6 +892,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("unused")) } + async fn reveal_secret( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + async fn inline_provider_refresh_secrets( &self, _: tonic::Request, diff --git a/crates/openshell-server/src/auth/audit/catalog.rs b/crates/openshell-server/src/auth/audit/catalog.rs index 9abdd81117..d850a3c2a8 100644 --- a/crates/openshell-server/src/auth/audit/catalog.rs +++ b/crates/openshell-server/src/auth/audit/catalog.rs @@ -10,6 +10,8 @@ const CREATE: ActivityId = ActivityId::Open; const UPDATE: ActivityId = ActivityId::Reset; const DELETE: ActivityId = ActivityId::Fail; const OTHER: ActivityId = ActivityId::Other; +/// API Activity Read (2). +const READ: ActivityId = ActivityId::Close; macro_rules! openshell { ($method:literal) => { @@ -50,6 +52,7 @@ const CATALOG: &[(&str, ActivityId)] = &[ (openshell!("CreateSecret"), CREATE), (openshell!("UpdateSecret"), UPDATE), (openshell!("DeleteSecret"), DELETE), + (openshell!("RevealSecret"), READ), (openshell!("UpdateConfig"), UPDATE), (openshell!("ApproveDraftChunk"), UPDATE), (openshell!("ApproveAllDraftChunks"), UPDATE), @@ -86,7 +89,13 @@ mod tests { .is_some_and(|scope| scope.ends_with(":write")) }) .collect(); - let cataloged: BTreeSet<&str> = CATALOG.iter().map(|(path, ..)| *path).collect(); + // Reads are audited only when they return secret values. + let audited_reads = BTreeSet::from([openshell!("RevealSecret")]); + let cataloged: BTreeSet<&str> = CATALOG + .iter() + .map(|(path, ..)| *path) + .filter(|path| !audited_reads.contains(path)) + .collect(); let missing: Vec<_> = write_scoped.difference(&cataloged).collect(); let stale: Vec<_> = cataloged.difference(&write_scoped).collect(); @@ -99,7 +108,7 @@ mod tests { "audit catalog names RPCs that are not write-scoped: {stale:?}" ); assert_eq!( - cataloged.len(), + cataloged.len() + audited_reads.len(), CATALOG.len(), "audit catalog lists an RPC twice" ); diff --git a/crates/openshell-server/src/cli.rs b/crates/openshell-server/src/cli.rs index f3f98c5ada..afedb09ef8 100644 --- a/crates/openshell-server/src/cli.rs +++ b/crates/openshell-server/src/cli.rs @@ -356,6 +356,10 @@ struct RunArgs { /// them. See the workspace secrets activation guidance. #[arg(long, env = "OPENSHELL_ENABLE_WORKSPACE_SECRETS", default_value_t = false, action = ArgAction::Set)] enable_workspace_secrets: bool, + + /// Allow `RevealSecret` for operators and OIDC tokens with the `secret:reveal` scope. + #[arg(long, env = "OPENSHELL_ENABLE_SECRET_REVEAL", default_value_t = false, action = ArgAction::Set)] + enable_secret_reveal: bool, } pub fn command() -> Command { @@ -664,7 +668,8 @@ fn prepare_server_config_with_drivers( .map_err(|error| miette::miette!("invalid service routing configuration: {error}"))?; config.enable_sandbox_updates = args.enable_sandbox_updates; - config.enable_workspace_secrets = args.enable_workspace_secrets; + config.workspace_secrets.enabled = args.enable_workspace_secrets; + config.workspace_secrets.reveal_enabled = args.enable_secret_reveal; config.supervisor_app_listener = file .as_ref() @@ -1624,6 +1629,11 @@ fn merge_file_into_args(args: &mut RunArgs, file: &GatewayFileSection, matches: { args.enable_workspace_secrets = enabled; } + if let Some(enabled) = file.enable_secret_reveal + && arg_defaulted(matches, "enable_secret_reveal") + { + args.enable_secret_reveal = enabled; + } if let Some(mtls_auth) = &file.mtls_auth && arg_defaulted(matches, "enable_mtls_auth") { diff --git a/crates/openshell-server/src/config_file.rs b/crates/openshell-server/src/config_file.rs index 576659d098..1fdc6ae317 100644 --- a/crates/openshell-server/src/config_file.rs +++ b/crates/openshell-server/src/config_file.rs @@ -174,6 +174,9 @@ pub struct GatewayFileSection { /// Explicit activation only after every serving gateway supports workspace secrets. #[serde(default)] pub enable_workspace_secrets: Option, + /// Allow control-plane callers to read workspace secret values. + #[serde(default)] + pub enable_secret_reveal: Option, #[serde(default)] pub grpc_rate_limit_requests: Option, #[serde(default)] diff --git a/crates/openshell-server/src/grpc/mod.rs b/crates/openshell-server/src/grpc/mod.rs index e14a54e585..6275d85efb 100644 --- a/crates/openshell-server/src/grpc/mod.rs +++ b/crates/openshell-server/src/grpc/mod.rs @@ -76,7 +76,8 @@ use openshell_core::proto::{ use openshell_core::proto::{ CreateSecretRequest, DeleteSecretRequest, DeleteSecretResponse, GetSecretRequest, InlineProviderRefreshSecretsRequest, InlineProviderRefreshSecretsResponse, ListSecretsRequest, - ListSecretsResponse, SecretResponse, UpdateSecretRequest, + ListSecretsResponse, RevealSecretRequest, RevealSecretResponse, SecretResponse, + UpdateSecretRequest, }; use serde::{Deserialize, Serialize}; use std::collections::BTreeMap; @@ -894,6 +895,20 @@ impl OpenShell for OpenShellService { mutation_audit::audited(targets, mutation_replay::run(&self.state, request)).await } + async fn reveal_secret( + &self, + request: Request, + ) -> Result, Status> { + let targets = { + let request = request.get_ref(); + mutation_audit::scoped( + request.workspace_scope.as_ref(), + [mutation_audit::requested("secret", &request.name)], + ) + }; + mutation_audit::audited(targets, secret::handle_reveal_secret(&self.state, request)).await + } + // --- Config / Policy --- async fn get_sandbox_config( diff --git a/crates/openshell-server/src/grpc/mutation_audit.rs b/crates/openshell-server/src/grpc/mutation_audit.rs index b9626b417b..44f27a7de3 100644 --- a/crates/openshell-server/src/grpc/mutation_audit.rs +++ b/crates/openshell-server/src/grpc/mutation_audit.rs @@ -130,6 +130,10 @@ impl Audited for SandboxTemplateResponse { } } +/// Records who read which secret. The named target comes from the request; +/// the response carries the value and contributes nothing to the record. +impl Audited for openshell_core::proto::RevealSecretResponse {} + impl Audited for SecretResponse { fn produced(&self) -> Vec> { stored( diff --git a/crates/openshell-server/src/grpc/secret.rs b/crates/openshell-server/src/grpc/secret.rs index 61065165a8..7d7ee78e57 100644 --- a/crates/openshell-server/src/grpc/secret.rs +++ b/crates/openshell-server/src/grpc/secret.rs @@ -13,8 +13,8 @@ use std::sync::Arc; use openshell_core::proto::datamodel::v1::{CredentialHandle, ObjectMeta}; use openshell_core::proto::{ CreateSecretRequest, DeleteSecretRequest, DeleteSecretResponse, DeletionOutcome, - GetSecretRequest, ListSecretsRequest, ListSecretsResponse, Secret, SecretResponse, - UpdateSecretRequest, + GetSecretRequest, ListSecretsRequest, ListSecretsResponse, RevealSecretRequest, + RevealSecretResponse, Secret, SecretResponse, UpdateSecretRequest, }; use openshell_core::{GetResourceVersion, ObjectId, ObjectLabels, ObjectName, ObjectWorkspace}; use openshell_credential_issuer::endpoint::normalize_token_endpoint; @@ -27,6 +27,8 @@ use tracing::warn; use super::validation::validate_dns1123_label; use super::{MAX_MAP_VALUE_LEN, MAX_NAME_LEN}; use crate::ServerState; +use crate::auth::identity::IdentityProvider; +use crate::auth::principal::Principal; use crate::auth::workspace_authz::{ MinWorkspaceRole, authorize_workspace, selected_workspace_name, }; @@ -88,7 +90,7 @@ fn labels_json(stored: &StoredWorkspaceSecret) -> Result, Status> /// Reject secret creation and new references unless the operator activated /// workspace secrets and no observable live gateway lacks support for them. pub async fn ensure_workspace_secrets_active(state: &ServerState) -> Result<(), Status> { - if !state.config.enable_workspace_secrets { + if !state.config.workspace_secrets.enabled { return Err(Status::failed_precondition( "workspace secrets are not enabled; set enable_workspace_secrets after every gateway supports them", )); @@ -189,7 +191,7 @@ fn validate_labels(labels: &HashMap) -> Result<(), Status> { async fn authorized_workspace( state: &ServerState, - request_principal: &crate::auth::principal::Principal, + request_principal: &Principal, workspace_scope: Option<&openshell_core::proto::WorkspaceSelector>, role: MinWorkspaceRole, ) -> Result { @@ -572,6 +574,88 @@ pub(super) async fn handle_delete_secret( })) } +/// The only scope that authorizes `RevealSecret`. `openshell:all` does not +/// imply it. +const REVEAL_SCOPE: &str = "secret:reveal"; + +/// Fail closed regardless of how the gateway authenticates users: middleware +/// skips scope checks when `scopes_claim` is unset and treats an empty admin +/// role as platform admin, so neither may grant a reveal here. +fn ensure_reveal_principal(state: &ServerState, principal: &Principal) -> Result<(), Status> { + match principal { + Principal::Operator(_) => Ok(()), + Principal::User(user) => { + let enforces_scopes = state + .config + .oidc + .as_ref() + .is_some_and(|oidc| !oidc.scopes_claim.is_empty()); + if user.identity.provider == IdentityProvider::Oidc + && enforces_scopes + && user + .identity + .scopes + .iter() + .any(|scope| scope == REVEAL_SCOPE) + { + Ok(()) + } else { + Err(Status::permission_denied(format!( + "revealing a secret requires an operator, or an OIDC token with the literal '{REVEAL_SCOPE}' scope on a gateway that enforces scopes" + ))) + } + } + _ => Err(Status::permission_denied( + "this principal may not reveal secrets", + )), + } +} + +pub(super) async fn handle_reveal_secret( + state: &Arc, + request: Request, +) -> Result, Status> { + let principal = super::extract_principal(&request)?; + let req = request.into_inner(); + if !state.config.workspace_secrets.reveal_enabled { + return Err(Status::failed_precondition( + "secret reveal is not enabled; set enable_secret_reveal for control-plane callers", + )); + } + ensure_reveal_principal(state, &principal)?; + if req.name.is_empty() { + return Err(Status::invalid_argument("name is required")); + } + let workspace = authorized_workspace( + state, + &principal, + req.workspace_scope.as_ref(), + MinWorkspaceRole::Admin, + ) + .await? + .name; + let stored = get_live_secret(&state.store, &workspace, &req.name) + .await? + .ok_or_else(|| Status::not_found("secret not found"))?; + let handle = stored + .value_handle + .clone() + .ok_or_else(|| Status::not_found("secret not found"))?; + let resolved = state + .credentials + .resolve_secrets(owner(&stored), &value_handles(handle), 0) + .await?; + let value = resolved + .values + .get(SECRET_VALUE_KEY) + .cloned() + .ok_or_else(|| Status::unavailable("secret value could not be read"))?; + Ok(Response::new(RevealSecretResponse { + value, + value_update_time: stored.value_update_time, + })) +} + /// Live refresh configurations in a workspace that reference a secret, named /// `provider/credential_key`. A malformed binding annotation counts as a /// reference so deletion fails closed. @@ -744,7 +828,8 @@ mod tests { Arc::get_mut(&mut state) .expect("test state is not shared yet") .config - .enable_workspace_secrets = true; + .workspace_secrets + .enabled = true; state } @@ -1012,4 +1097,115 @@ mod tests { assert!(stored(&state, "acme").await.is_none()); assert!(resolve_value(&state, &tombstoned, &handle).await.is_err()); } + + fn oidc_config(scopes_claim: &str) -> openshell_core::OidcConfig { + openshell_core::OidcConfig { + issuer: "https://issuer.example.com".to_string(), + ca_bundle: None, + dangerously_allow_insecure_http: false, + jwks_allowed_origins: Vec::new(), + audience: "openshell".to_string(), + jwks_ttl_secs: 3600, + roles_claim: "roles".to_string(), + admin_role: String::new(), + user_role: String::new(), + scopes_claim: scopes_claim.to_string(), + } + } + + async fn reveal_state(scopes_claim: Option<&str>) -> Arc { + let mut state = enabled_state().await; + create(&state, "acme", "the-value").await; + let config = &mut Arc::get_mut(&mut state) + .expect("test state is not shared yet") + .config; + config.workspace_secrets.reveal_enabled = true; + config.oidc = scopes_claim.map(oidc_config); + state + } + + fn user(provider: IdentityProvider, scopes: &[&str]) -> Principal { + Principal::User(crate::auth::principal::UserPrincipal { + identity: crate::auth::identity::Identity { + subject: "occ".to_string(), + display_name: None, + roles: vec!["openshell-admin".to_string()], + scopes: scopes.iter().map(ToString::to_string).collect(), + provider, + }, + }) + } + + async fn reveal(state: &Arc, principal: Principal) -> Result { + let mut request = Request::new(RevealSecretRequest { + workspace_scope: Some(workspace_selector("default".to_string())), + name: "acme".to_string(), + }); + request.extensions_mut().insert(principal); + handle_reveal_secret(state, request) + .await + .map(|response| response.into_inner().value) + } + + #[tokio::test] + async fn reveal_is_disabled_by_default() { + let state = enabled_state().await; + create(&state, "acme", "the-value").await; + let err = reveal(&state, user(IdentityProvider::Oidc, &[REVEAL_SCOPE])) + .await + .unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + } + + #[tokio::test] + async fn reveal_requires_the_literal_scope_on_a_scope_enforcing_gateway() { + let state = reveal_state(Some("scope")).await; + assert_eq!( + reveal(&state, user(IdentityProvider::Oidc, &[REVEAL_SCOPE])) + .await + .unwrap(), + "the-value" + ); + for principal in [ + user(IdentityProvider::Oidc, &["openshell:all"]), + user(IdentityProvider::Oidc, &["secret:read", "secret:write"]), + user(IdentityProvider::Mtls, &[REVEAL_SCOPE]), + user(IdentityProvider::LocalDev, &[REVEAL_SCOPE]), + ] { + assert_eq!( + reveal(&state, principal).await.unwrap_err().code(), + Code::PermissionDenied + ); + } + } + + #[tokio::test] + async fn reveal_fails_closed_when_scopes_are_not_enforced() { + for scopes_claim in [None, Some("")] { + let state = reveal_state(scopes_claim).await; + assert_eq!( + reveal(&state, user(IdentityProvider::Oidc, &[REVEAL_SCOPE])) + .await + .unwrap_err() + .code(), + Code::PermissionDenied + ); + } + } + + #[tokio::test] + async fn operators_may_reveal() { + let state = reveal_state(None).await; + let operator = Principal::Operator(crate::auth::principal::OperatorPrincipal { + identity: crate::auth::identity::Identity { + subject: "operator".to_string(), + display_name: None, + roles: vec!["operator".to_string()], + scopes: Vec::new(), + provider: IdentityProvider::Mtls, + }, + certificate_sha256: "00".to_string(), + }); + assert_eq!(reveal(&state, operator).await.unwrap(), "the-value"); + } } diff --git a/crates/openshell-server/src/grpc/secret_refresh_tests.rs b/crates/openshell-server/src/grpc/secret_refresh_tests.rs index 40d5d824d7..2cf22fa82e 100644 --- a/crates/openshell-server/src/grpc/secret_refresh_tests.rs +++ b/crates/openshell-server/src/grpc/secret_refresh_tests.rs @@ -44,7 +44,8 @@ async fn enabled_state() -> Arc { Arc::get_mut(&mut state) .expect("test state is not shared yet") .config - .enable_workspace_secrets = true; + .workspace_secrets + .enabled = true; state } @@ -452,7 +453,8 @@ async fn references_require_activation() { Arc::get_mut(&mut state) .expect("test state is not shared") .config - .enable_workspace_secrets = false; + .workspace_secrets + .enabled = false; let err = configure( &state, configure_request(ProviderCredentialRefreshStrategy::Oauth2ClientCredentials), diff --git a/crates/openshell-server/src/storage_proto.rs b/crates/openshell-server/src/storage_proto.rs index 116af51f45..6751f208fb 100644 --- a/crates/openshell-server/src/storage_proto.rs +++ b/crates/openshell-server/src/storage_proto.rs @@ -226,7 +226,7 @@ mod tests { // updates use a separate capability/service and add one classified RPC. // Discovery RPCs move to ServiceDiscovery; message and durable closures are unchanged. const PUBLIC_RPC_SCHEMA_SHA256: &str = - "2cd18038b7655d784ad2819eabc1d602d2690f887378282921872277d1a91dc6"; + "f069782f09e8763fdf6041eb7e34ab00b63066d283dc1b16b78584b6d1d7bfef"; const DURABLE_SCHEMA_SHA256: &str = "b6a1de55ca3bdf2ccbf88a32f57e87db84b775c48fcd5f9fcf7811eae8d67c70"; const PUBLIC_DURABLE_OVERLAP_SHA256: &str = @@ -703,7 +703,7 @@ mod tests { } assert_eq!( compiled_method_count, - 119 + PROVIDER_READINESS_RPC_SIGNATURES.len() + 120 + PROVIDER_READINESS_RPC_SIGNATURES.len() + PEER_OWNER_RPC_SIGNATURES.len() + PEER_CONFIG_RPC_SIGNATURES.len() + isolation_methods.len() @@ -713,7 +713,7 @@ mod tests { ); assert_eq!( methods.len(), - 89 + PROVIDER_READINESS_RPC_SIGNATURES.len() + 90 + PROVIDER_READINESS_RPC_SIGNATURES.len() + PEER_OWNER_RPC_SIGNATURES.len() + PEER_CONFIG_RPC_SIGNATURES.len(), "inventory every public gateway RPC" @@ -723,7 +723,7 @@ mod tests { .iter() .filter(|method| method.starts_with("openshell.v1.OpenShell/")) .count(), - 86 + PROVIDER_READINESS_RPC_SIGNATURES.len() + 87 + PROVIDER_READINESS_RPC_SIGNATURES.len() + PEER_OWNER_RPC_SIGNATURES.len() + PEER_CONFIG_RPC_SIGNATURES.len() ); @@ -774,7 +774,7 @@ mod tests { overlap_hash.as_str(), ), ( - (360, 31), + (362, 31), (101, 22), (86, 22), PUBLIC_RPC_SCHEMA_SHA256, diff --git a/crates/openshell-server/tests/common/mod.rs b/crates/openshell-server/tests/common/mod.rs index ff40ab1ed4..2bce0f3bb8 100644 --- a/crates/openshell-server/tests/common/mod.rs +++ b/crates/openshell-server/tests/common/mod.rs @@ -449,6 +449,13 @@ impl OpenShell for TestOpenShell { Err(Status::unimplemented("not implemented in test")) } + async fn reveal_secret( + &self, + _request: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("not implemented in test")) + } + async fn inline_provider_refresh_secrets( &self, _request: tonic::Request, diff --git a/crates/openshell-server/tests/supervisor_relay_integration.rs b/crates/openshell-server/tests/supervisor_relay_integration.rs index 8e20e12887..178ffeb826 100644 --- a/crates/openshell-server/tests/supervisor_relay_integration.rs +++ b/crates/openshell-server/tests/supervisor_relay_integration.rs @@ -477,6 +477,13 @@ impl OpenShell for RelayGateway { Err(Status::unimplemented("unused")) } + async fn reveal_secret( + &self, + _: tonic::Request, + ) -> Result, Status> { + Err(Status::unimplemented("unused")) + } + async fn inline_provider_refresh_secrets( &self, _: tonic::Request, diff --git a/proto/openshell.proto b/proto/openshell.proto index 7cc3ef68f3..266adb5307 100644 --- a/proto/openshell.proto +++ b/proto/openshell.proto @@ -509,6 +509,19 @@ service OpenShell { }; } + // Return a workspace secret's plaintext value to a control-plane caller. + // Disabled unless the gateway sets enable_secret_reveal. Callers must be an + // operator, or an OIDC workspace admin whose token carries the literal + // secret:reveal scope on a gateway that enforces scopes; openshell:all does + // not satisfy it. Every call is audited. + rpc RevealSecret(RevealSecretRequest) returns (RevealSecretResponse) { + option (openshell.options.v1.authorization) = { + auth_mode: "bearer" + scope: "secret:reveal" + workspace_role: "admin" + }; + } + // Get sandbox settings by id (called by users, and by supervisors at startup // and in their poll loop). A supervisor whose session pushes configuration // receives later updates over ConnectSupervisor and stops polling. @@ -2953,6 +2966,18 @@ message DeleteSecretResponse { DeletionOutcome outcome = 1; } +message RevealSecretRequest { + // Workspace scope. Only a named workspace selection is accepted. + openshell.datamodel.v1.WorkspaceSelector workspace_scope = 1; + string name = 2; +} + +message RevealSecretResponse { + string value = 1 [(openshell.options.v1.secret) = true]; + // Last time the value was written. + google.protobuf.Timestamp value_update_time = 2; +} + // Get sandbox provider environment request. message GetSandboxProviderEnvironmentRequest { // The sandbox ID. From 44e759841c4600ef930ce0ebc81c73e191620810 Mon Sep 17 00:00:00 2001 From: Mrunal Patel Date: Wed, 18 Nov 2026 08:24:45 +0000 Subject: [PATCH 05/10] feat(cli): add workspace secret commands Add openshell secret create, update, get, list, and delete. Values come from an environment variable or a file and are never printed. Add --material-secret to provider refresh configure, which also lists each referenced key in secret_material_keys, and provider refresh inline-secrets for unwinding references. Refs #5563 Signed-off-by: Mrunal Patel --- crates/openshell-cli/src/commands/mod.rs | 1 + crates/openshell-cli/src/commands/secret.rs | 355 ++++++++++++++++++++ crates/openshell-cli/src/main.rs | 208 ++++++++++++ crates/openshell-cli/src/run.rs | 4 + 4 files changed, 568 insertions(+) create mode 100644 crates/openshell-cli/src/commands/secret.rs diff --git a/crates/openshell-cli/src/commands/mod.rs b/crates/openshell-cli/src/commands/mod.rs index 5158a4ac5d..e2bb38fdb0 100644 --- a/crates/openshell-cli/src/commands/mod.rs +++ b/crates/openshell-cli/src/commands/mod.rs @@ -5,3 +5,4 @@ pub mod common; pub mod gateway; pub mod provider; pub mod provider_readiness; +pub mod secret; diff --git a/crates/openshell-cli/src/commands/secret.rs b/crates/openshell-cli/src/commands/secret.rs new file mode 100644 index 0000000000..8dbdd0cc6c --- /dev/null +++ b/crates/openshell-cli/src/commands/secret.rs @@ -0,0 +1,355 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +//! `openshell secret`: workspace secrets used as shared minting material. +//! +//! Values are read from an environment variable or a file, never from a +//! command-line argument, and the gateway never returns them to this CLI. + +use crate::color::Colorize; +use crate::commands::common::{format_epoch_ms, parse_key_value_pairs}; +use crate::tls::{TlsOptions, grpc_client}; +use miette::{IntoDiagnostic, Result, WrapErr, miette}; +use openshell_core::proto::{ + CreateSecretRequest, DeleteSecretRequest, GetSecretRequest, ListSecretsRequest, Secret, + SecretLabels, UpdateSecretRequest, workspace_selector, +}; +use std::path::Path; + +/// Where a secret value comes from. At most one source may be given. +#[derive(Debug, Clone, Copy, Default)] +pub struct SecretValueSource<'a> { + pub env: Option<&'a str>, + pub file: Option<&'a Path>, +} + +impl SecretValueSource<'_> { + fn read(self) -> Result> { + match (self.env, self.file) { + (Some(_), Some(_)) => Err(miette!("pass only one of --value-env and --value-file")), + (Some(var), None) => { + let value = std::env::var(var) + .map_err(|_| miette!("--value-env requires local env var '{var}' to be set"))?; + if value.is_empty() { + return Err(miette!("local env var '{var}' is empty")); + } + Ok(Some(value)) + } + (None, Some(path)) => { + let mut value = std::fs::read_to_string(path) + .into_diagnostic() + .wrap_err_with(|| format!("failed to read {}", path.display()))?; + // Files written by editors and `echo` end with a newline that + // is not part of the secret. + if value.ends_with('\n') { + value.pop(); + if value.ends_with('\r') { + value.pop(); + } + } + if value.is_empty() { + return Err(miette!("{} is empty", path.display())); + } + Ok(Some(value)) + } + (None, None) => Ok(None), + } + } +} + +pub struct SecretCreateInput<'a> { + pub name: &'a str, + pub description: &'a str, + pub value: SecretValueSource<'a>, + pub allowed_endpoints: &'a [String], + pub labels: &'a [String], +} + +pub async fn secret_create( + server: &str, + input: SecretCreateInput<'_>, + output: &str, + workspace: &str, + tls: &TlsOptions, +) -> Result<()> { + let value = input + .value + .read()? + .ok_or_else(|| miette!("pass --value-env or --value-file"))?; + let labels = parse_key_value_pairs(input.labels, "--label")?; + let mut client = grpc_client(server, tls).await?; + let secret = client + .create_secret(CreateSecretRequest { + workspace_scope: Some(workspace_selector(workspace.to_string())), + name: input.name.to_string(), + description: input.description.to_string(), + value, + allowed_endpoints: input.allowed_endpoints.to_vec(), + labels, + request_id: String::new(), + }) + .await + .into_diagnostic()? + .into_inner() + .secret + .ok_or_else(|| miette!("secret missing from response"))?; + if crate::output::print_output_single(output, &secret, secret_to_json)? { + return Ok(()); + } + println!( + "{} Created secret {}", + "✓".green().bold(), + secret_name(&secret) + ); + Ok(()) +} + +pub struct SecretUpdateInput<'a> { + pub name: &'a str, + pub value: SecretValueSource<'a>, + pub description: Option<&'a str>, + pub labels: Option<&'a [String]>, +} + +pub async fn secret_update( + server: &str, + input: SecretUpdateInput<'_>, + output: &str, + workspace: &str, + tls: &TlsOptions, +) -> Result<()> { + let value = input.value.read()?; + let labels = input + .labels + .map(|labels| parse_key_value_pairs(labels, "--label")) + .transpose()? + .map(|labels| SecretLabels { labels }); + if value.is_none() && input.description.is_none() && labels.is_none() { + return Err(miette!( + "nothing to update; pass --value-env, --value-file, --description, or --label" + )); + } + let mut client = grpc_client(server, tls).await?; + let secret = client + .update_secret(UpdateSecretRequest { + workspace_scope: Some(workspace_selector(workspace.to_string())), + name: input.name.to_string(), + value, + description: input.description.map(str::to_string), + labels, + expected_resource_version: None, + request_id: String::new(), + }) + .await + .into_diagnostic()? + .into_inner() + .secret + .ok_or_else(|| miette!("secret missing from response"))?; + if crate::output::print_output_single(output, &secret, secret_to_json)? { + return Ok(()); + } + println!( + "{} Updated secret {}", + "✓".green().bold(), + secret_name(&secret) + ); + Ok(()) +} + +pub async fn secret_get( + server: &str, + name: &str, + output: &str, + workspace: &str, + tls: &TlsOptions, +) -> Result<()> { + let mut client = grpc_client(server, tls).await?; + let secret = client + .get_secret(GetSecretRequest { + workspace_scope: Some(workspace_selector(workspace.to_string())), + name: name.to_string(), + }) + .await + .into_diagnostic()? + .into_inner() + .secret + .ok_or_else(|| miette!("secret missing from response"))?; + if crate::output::print_output_single(output, &secret, secret_to_json)? { + return Ok(()); + } + println!("{}", "Secret:".cyan().bold()); + println!(); + println!(" {} {}", "Name:".dimmed(), secret_name(&secret)); + if !secret.description.is_empty() { + println!(" {} {}", "Description:".dimmed(), secret.description); + } + println!( + " {} {}", + "Allowed endpoints:".dimmed(), + secret.allowed_endpoints.join(", ") + ); + println!(" {} {}", "Value updated:".dimmed(), value_updated(&secret)); + if let Some(meta) = &secret.metadata + && !meta.labels.is_empty() + { + let mut labels: Vec<_> = meta + .labels + .iter() + .map(|(k, v)| format!("{k}={v}")) + .collect(); + labels.sort(); + println!(" {} {}", "Labels:".dimmed(), labels.join(", ")); + } + Ok(()) +} + +pub async fn secret_list( + server: &str, + page_size: i32, + page_token: &str, + label_selector: &str, + output: &str, + workspace: &str, + tls: &TlsOptions, +) -> Result<()> { + let mut client = grpc_client(server, tls).await?; + let response = client + .list_secrets(ListSecretsRequest { + workspace_scope: Some(workspace_selector(workspace.to_string())), + page_size, + page_token: page_token.to_string(), + label_selector: label_selector.to_string(), + }) + .await + .into_diagnostic()? + .into_inner(); + if crate::output::print_paginated_output_collection( + output, + "secrets", + &response.secrets, + &response.next_page_token, + secret_to_json, + )? { + return Ok(()); + } + crate::output::print_next_page_token(&response.next_page_token); + if response.secrets.is_empty() { + println!("No secrets found."); + return Ok(()); + } + let name_width = response + .secrets + .iter() + .map(|secret| secret_name(secret).len()) + .max() + .unwrap_or(4) + .max(4); + println!( + "{: Result<()> { + let mut client = grpc_client(server, tls).await?; + for name in names { + let response = client + .delete_secret(DeleteSecretRequest { + workspace_scope: Some(workspace_selector(workspace.to_string())), + name: name.clone(), + allow_missing: true, + request_id: String::new(), + }) + .await + .into_diagnostic()? + .into_inner(); + if crate::run::deletion_completed(response.outcome)? { + println!("{} Deleted secret {name}", "✓".green().bold()); + } else { + println!("Secret '{name}' not found."); + } + } + Ok(()) +} + +fn secret_name(secret: &Secret) -> &str { + secret + .metadata + .as_ref() + .map_or("", |meta| meta.name.as_str()) +} + +fn value_updated(secret: &Secret) -> String { + secret + .value_update_time + .as_ref() + .and_then(|time| openshell_core::time::timestamp_to_millis(time).ok()) + .map_or_else(|| "-".to_string(), format_epoch_ms) +} + +fn secret_to_json(secret: &Secret) -> serde_json::Value { + let meta = secret.metadata.clone().unwrap_or_default(); + serde_json::json!({ + "name": meta.name, + "id": meta.id, + "workspace": meta.workspace, + "resource_version": meta.resource_version, + "labels": meta.labels, + "description": secret.description, + "allowed_endpoints": secret.allowed_endpoints, + "value_updated_at": secret + .value_update_time + .as_ref() + .and_then(|time| openshell_core::time::timestamp_to_millis(time).ok()) + .map(format_epoch_ms), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn value_sources_are_exclusive_and_trim_one_trailing_newline() { + let dir = tempfile::tempdir().unwrap(); + let path = dir.path().join("secret"); + std::fs::write(&path, "s3cret\r\n").unwrap(); + let file = SecretValueSource { + env: None, + file: Some(&path), + }; + assert_eq!(file.read().unwrap().as_deref(), Some("s3cret")); + + let both = SecretValueSource { + env: Some("OPENSHELL_UNUSED"), + file: Some(&path), + }; + assert!(both.read().is_err()); + assert!(SecretValueSource::default().read().unwrap().is_none()); + + std::fs::write(&path, "\n").unwrap(); + assert!(file.read().is_err(), "an empty value is rejected"); + } + + #[test] + fn json_output_never_carries_a_value_field() { + let json = secret_to_json(&Secret::default()); + assert!(json.get("value").is_none()); + } +} diff --git a/crates/openshell-cli/src/main.rs b/crates/openshell-cli/src/main.rs index 767c1f5b06..2bdd74613a 100644 --- a/crates/openshell-cli/src/main.rs +++ b/crates/openshell-cli/src/main.rs @@ -409,6 +409,14 @@ const WORKSPACE_EXAMPLES: &str = "\x1b[1mALIAS\x1b[0m $ openshell workspace delete staging "; +const SECRET_EXAMPLES: &str = "\x1b[1mEXAMPLES\x1b[0m + $ openshell secret create acme-oauth-client --value-env CLIENT_SECRET \\ + --allowed-endpoint https://login.example.com/oauth/token + $ openshell secret update acme-oauth-client --value-file ./client-secret + $ openshell secret list + $ openshell secret delete acme-oauth-client +"; + const GATEWAY_EXAMPLES: &str = "\x1b[1mALIAS\x1b[0m gw @@ -603,6 +611,13 @@ enum Commands { command: Option, }, + /// Manage workspace secrets used as shared credential minting material. + #[command(after_help = SECRET_EXAMPLES, help_template = SUBCOMMAND_HELP_TEMPLATE)] + Secret { + #[command(subcommand)] + command: Option, + }, + // =================================================================== // GATEWAY COMMANDS // =================================================================== @@ -2597,6 +2612,107 @@ enum WorkspaceCommands { Member(WorkspaceMemberCommands), } +#[derive(Subcommand, Debug)] +enum SecretCommands { + /// Create a workspace secret. + #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] + Create { + /// Secret name (DNS-1123 label). + name: String, + + /// Read the value from this local environment variable. + #[arg(long = "value-env", value_name = "ENVVAR")] + value_env: Option, + + /// Read the value from this file. One trailing newline is removed. + #[arg(long = "value-file", value_name = "PATH")] + value_file: Option, + + /// Token endpoint the value may be sent to. Repeatable; cannot be + /// changed after creation. + #[arg(long = "allowed-endpoint", value_name = "URL", required = true)] + allowed_endpoints: Vec, + + /// Human-readable description. + #[arg(long, default_value = "")] + description: String, + + /// Labels to attach to the secret (KEY=VALUE). + #[arg(long = "label", value_name = "KEY=VALUE")] + labels: Vec, + + /// Output format. + #[arg(short = 'o', long = "output", value_enum, default_value_t = OutputFormat::Table)] + output: OutputFormat, + }, + + /// Replace a workspace secret's value, description, or labels. + #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] + Update { + /// Secret name. + name: String, + + /// Read the new value from this local environment variable. + #[arg(long = "value-env", value_name = "ENVVAR")] + value_env: Option, + + /// Read the new value from this file. One trailing newline is removed. + #[arg(long = "value-file", value_name = "PATH")] + value_file: Option, + + /// New description. + #[arg(long)] + description: Option, + + /// Replace all labels (KEY=VALUE). Repeatable. + #[arg(long = "label", value_name = "KEY=VALUE")] + labels: Option>, + + /// Output format. + #[arg(short = 'o', long = "output", value_enum, default_value_t = OutputFormat::Table)] + output: OutputFormat, + }, + + /// Show workspace secret metadata. The value is never shown. + #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] + Get { + /// Secret name. + name: String, + + /// Output format. + #[arg(short = 'o', long = "output", value_enum, default_value_t = OutputFormat::Table)] + output: OutputFormat, + }, + + /// List workspace secrets. + #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] + List { + /// Maximum number of secrets to return in this page. + #[arg(long, default_value_t = 100)] + page_size: i32, + + /// Opaque continuation token from a previous page. + #[arg(long, default_value = "")] + page_token: String, + + /// Filter by label selector (e.g. "team=platform"). + #[arg(long)] + label_selector: Option, + + /// Output format. + #[arg(short = 'o', long = "output", value_enum, default_value_t = OutputFormat::Table)] + output: OutputFormat, + }, + + /// Delete workspace secrets. Fails while a provider refresh references one. + #[command(help_template = LEAF_HELP_TEMPLATE, next_help_heading = "FLAGS")] + Delete { + /// Secret name(s) to delete. + #[arg(required = true)] + names: Vec, + }, +} + #[derive(Subcommand, Debug)] enum WorkspaceMemberCommands { /// Add a member to a workspace. @@ -4294,6 +4410,98 @@ async fn run_async() -> Result<()> { .print_help() .expect("Failed to print help"); } + Some(Commands::Secret { + command: Some(command), + }) => { + let ctx = resolve_gateway(&cli.gateway, &cli.gateway_endpoint)?; + let endpoint = &ctx.endpoint; + let mut tls = tls.with_gateway_name(&ctx.name); + apply_auth(&mut tls, &ctx.name)?; + match command { + SecretCommands::Create { + name, + value_env, + value_file, + allowed_endpoints, + description, + labels, + output, + } => { + run::secret_create( + endpoint, + run::SecretCreateInput { + name: &name, + description: &description, + value: run::SecretValueSource { + env: value_env.as_deref(), + file: value_file.as_deref(), + }, + allowed_endpoints: &allowed_endpoints, + labels: &labels, + }, + output.as_str(), + &cli.workspace, + &tls, + ) + .await?; + } + SecretCommands::Update { + name, + value_env, + value_file, + description, + labels, + output, + } => { + run::secret_update( + endpoint, + run::SecretUpdateInput { + name: &name, + value: run::SecretValueSource { + env: value_env.as_deref(), + file: value_file.as_deref(), + }, + description: description.as_deref(), + labels: labels.as_deref(), + }, + output.as_str(), + &cli.workspace, + &tls, + ) + .await?; + } + SecretCommands::Get { name, output } => { + run::secret_get(endpoint, &name, output.as_str(), &cli.workspace, &tls).await?; + } + SecretCommands::List { + page_size, + page_token, + label_selector, + output, + } => { + run::secret_list( + endpoint, + page_size, + &page_token, + label_selector.as_deref().unwrap_or(""), + output.as_str(), + &cli.workspace, + &tls, + ) + .await?; + } + SecretCommands::Delete { names } => { + run::secret_delete(endpoint, &names, &cli.workspace, &tls).await?; + } + } + } + Some(Commands::Secret { command: None }) => { + Cli::command() + .find_subcommand_mut("secret") + .expect("secret subcommand exists") + .print_help() + .expect("Failed to print help"); + } Some(Commands::Workspace { command: None }) => { Cli::command() .find_subcommand_mut("workspace") diff --git a/crates/openshell-cli/src/run.rs b/crates/openshell-cli/src/run.rs index 9d12432ac9..653e7e58ae 100644 --- a/crates/openshell-cli/src/run.rs +++ b/crates/openshell-cli/src/run.rs @@ -34,6 +34,10 @@ pub use crate::commands::provider::{ sandbox_provider_attach, sandbox_provider_detach, sandbox_provider_list, }; pub use crate::commands::provider_readiness::{ProviderWaitOptions, sandbox_provider_status}; +pub use crate::commands::secret::{ + SecretCreateInput, SecretUpdateInput, SecretValueSource, secret_create, secret_delete, + secret_get, secret_list, secret_update, +}; use crate::color::Colorize; use crate::policy_update::build_policy_update_plan; From b8092113f6be941e9b69665d99590a7f59acfdbd Mon Sep 17 00:00:00 2001 From: Mrunal Patel Date: Wed, 18 Nov 2026 08:26:33 +0000 Subject: [PATCH 06/10] docs: document workspace secrets Describe shared secret material for provider refresh, the enable_workspace_secrets and enable_secret_reveal gateway settings with activation and rollback steps, and the new secret scopes. Refs #5563 Signed-off-by: Mrunal Patel --- docs/how-it-works/gateways/configuration.mdx | 38 +++++++++++++++ docs/how-it-works/providers/overview.mdx | 51 ++++++++++++++++++++ docs/how-it-works/workspaces.mdx | 6 ++- skills/openshell-cli/SKILL.md | 2 + 4 files changed, 95 insertions(+), 2 deletions(-) diff --git a/docs/how-it-works/gateways/configuration.mdx b/docs/how-it-works/gateways/configuration.mdx index 45f4f457fd..200815c6d3 100644 --- a/docs/how-it-works/gateways/configuration.mdx +++ b/docs/how-it-works/gateways/configuration.mdx @@ -305,6 +305,11 @@ log_level = "info" # Activate only after retiring incompatible gateways and outstanding driver work. enable_sandbox_updates = false +# Activate only after every gateway that can serve traffic supports workspace secrets. +enable_workspace_secrets = false +# Let operators and OIDC tokens with the secret:reveal scope read secret values. +enable_secret_reveal = false + # Connection ceiling for the gateway's database pool. Omit to use the backend # default: 10 for Postgres, 5 for on-disk SQLite. database_max_connections = 10 @@ -681,6 +686,39 @@ an image update is pending or retained sandboxes require the private image revision protocol. SQLite supports one gateway process only; activation attests that the previous process has stopped, not that sharing a SQLite file enables HA. +## Workspace Secrets + +`[openshell.gateway] enable_workspace_secrets` defaults to `false`. It gates +creating [workspace secrets](/how-it-works/providers/overview#shared-secret-material) +and referencing them from provider refresh configuration. Reading, updating, +and deleting existing secrets do not need it, and neither does +`provider refresh inline-secrets`. The equivalent gateway process flag and +environment variable are `--enable-workspace-secrets true` and +`OPENSHELL_ENABLE_WORKSPACE_SECRETS=true`. + +Upgrade every gateway before activation. Gateways that predate workspace +secrets reject requests that reference one, but they can delete a workspace +that holds secrets. With PostgreSQL, a request is refused while any live +gateway membership record lacks workspace-secret support. As with sandbox +updates, that check is not proof that older gateways have stopped; setting the +flag attests that they are retired. + +Before rolling back to a release without workspace secrets: + +1. Set `enable_workspace_secrets = false`. +2. For each credential whose `provider refresh status` lists a secret, run + `openshell provider refresh inline-secrets`. +3. Delete every workspace secret. + +`[openshell.gateway] enable_secret_reveal` defaults to `false`. When `true`, +the `RevealSecret` RPC returns secret values to operators authenticated with +operator mTLS, and to OIDC workspace admins whose token carries the literal +`secret:reveal` scope on a gateway with `scopes_claim` set. `openshell:all` +does not grant it, and the CLI has no reveal command. Every reveal is audited. +Enable it only for a control-plane service that performs its own OAuth +authorization-code exchange. The equivalent flag and environment variable are +`--enable-secret-reveal true` and `OPENSHELL_ENABLE_SECRET_REVEAL=true`. + ## Azure PostgreSQL workload identity Set `database_auth = "azure-workload-identity"` under `[openshell.gateway]` to diff --git a/docs/how-it-works/providers/overview.mdx b/docs/how-it-works/providers/overview.mdx index 5cb1f82e94..51cc53b7a3 100644 --- a/docs/how-it-works/providers/overview.mdx +++ b/docs/how-it-works/providers/overview.mdx @@ -324,6 +324,57 @@ Force a gateway-managed refresh for one credential: openshell provider refresh rotate my-graph --credential-key MS_GRAPH_ACCESS_TOKEN ``` +### Shared Secret Material + +Use a workspace secret when many providers share one secret, such as the +client secret of a confidential OAuth client that every user's provider +refreshes through. The gateway reads the secret's current value at each mint, +so updating the secret reaches every provider that references it without +reauthorizing users. Workspace secrets are minting material only: OpenShell +never injects them into sandboxes, and `openshell secret get` and +`openshell secret list` never show their values. + +Create the secret with the token endpoints it may be sent to: + +```shell +openshell secret create acme-oauth-client \ + --value-env CLIENT_SECRET \ + --allowed-endpoint https://login.example.com/oauth/token +``` + +Reference it from refresh configuration with `--material-secret KEY=SECRET`: + +```shell +openshell provider refresh configure alice-mcp \ + --credential-key MCP_ACCESS_TOKEN \ + --strategy oauth2-refresh-token \ + --material client_id="$CLIENT_ID" \ + --secret-material-env refresh_token=REFRESH_TOKEN \ + --material-secret client_secret=acme-oauth-client +``` + +Rotate the shared value with `openshell secret update acme-oauth-client +--value-file ./client-secret`. Each provider uses the new value at its next +refresh; `openshell provider refresh rotate` applies it immediately. + +These rules apply: + +- Only secret material can come from a workspace secret, and never a + `refresh_token`, which each provider rotates on its own. +- The token endpoint the refresh calls must be one of the secret's allowed + endpoints, and the gateway does not follow redirects for these requests. + Allowed endpoints cannot change; create a new secret for a new endpoint. +- Workspace secrets work with the OAuth2 and Google service account + strategies, not `aws_sts_assume_role`. +- `openshell secret delete` fails while a refresh references the secret. Run + `openshell provider refresh inline-secrets` to copy the current value into + the provider's own refresh material first. It keeps the current grant, + including a rotated refresh token. + +`provider refresh status` reports which secrets a credential references. The +gateway operator must enable workspace secrets with `enable_workspace_secrets`; +refer to [Gateway Configuration](/how-it-works/gateways/configuration#workspace-secrets). + ### AWS STS The gateway calls `sts:AssumeRole` and writes three short-lived credentials diff --git a/docs/how-it-works/workspaces.mdx b/docs/how-it-works/workspaces.mdx index 1789d0108a..da34737def 100644 --- a/docs/how-it-works/workspaces.mdx +++ b/docs/how-it-works/workspaces.mdx @@ -39,8 +39,10 @@ cannot perform platform-scoped or cross-workspace operations. When the gateway enables scope enforcement through `scopes_claim`, the token must also contain the scope required by the operation. Common scopes include `workspace:read`, `workspace:write`, `sandbox:read`, `sandbox:write`, -`provider:read`, `provider:write`, `config:read`, and `config:write`. -`openshell:all` satisfies every scope requirement. For OIDC and scope +`provider:read`, `provider:write`, `config:read`, `config:write`, +`secret:read`, and `secret:write`. +`openshell:all` satisfies every scope requirement except `secret:reveal`, +which a token must carry explicitly. For OIDC and scope configuration, refer to [Gateway Authentication](/how-it-works/gateways/authentication). The following table summarizes common operations. diff --git a/skills/openshell-cli/SKILL.md b/skills/openshell-cli/SKILL.md index 82e1000395..379e1fc659 100644 --- a/skills/openshell-cli/SKILL.md +++ b/skills/openshell-cli/SKILL.md @@ -245,6 +245,8 @@ openshell provider refresh rotate my-provider --credential-key ACCESS_TOKEN Prefer `--secret-material-env KEY[=ENVVAR]` for secret refresh material. `--material KEY=VALUE` is for non-secret material; `--secret-material-key` marks supplied material keys as secret. +When many providers share one OAuth client secret, store it once as a workspace secret and reference it with `--material-secret client_secret=NAME`. Rotating it with `openshell secret update` reaches every referencing provider at its next refresh. A secret only reaches the token endpoints it was created with (`--allowed-endpoint`), and `openshell secret delete` fails while a refresh references it; run `openshell provider refresh inline-secrets` to copy the value into the provider first. The gateway operator must set `enable_workspace_secrets`. See `openshell secret --help`. + The gateway stores secret refresh material through its active credential driver. With Vault selected, refresh tokens, client secrets, and private keys live in Vault alongside injectable provider credentials; refresh state contains only From bee4038e686d3c344ad3f70ed787b236c97a7503 Mon Sep 17 00:00:00 2001 From: Mrunal Patel Date: Wed, 18 Nov 2026 15:28:57 +0000 Subject: [PATCH 07/10] fix(server): close workspace secret races found in review - Refuse InlineProviderRefreshSecrets while a mint is in progress or committing, so replacing the refresh state cannot make the mint discard a rotated refresh token. A mint that starts later still loses the version-checked write to it. - Re-read the workspace after inserting a secret and undo the insert if the workspace was deleted meanwhile, so no secret or staged value outlives it. - Page the cleanup sweep through every secret instead of rereading the first 1000, so later tombstones and superseded handles are reached. - Reject configure-time keys such as refresh_before_seconds as secret references even when a profile marks them secret. - Show workspace-secret bindings in provider refresh status output. Refs #5563 Signed-off-by: Mrunal Patel --- crates/openshell-cli/src/commands/provider.rs | 36 +++- crates/openshell-server/src/grpc/provider.rs | 31 ++++ crates/openshell-server/src/grpc/secret.rs | 158 +++++++++++++++++- .../src/grpc/secret_refresh_tests.rs | 141 +++++++++++++++- 4 files changed, 355 insertions(+), 11 deletions(-) diff --git a/crates/openshell-cli/src/commands/provider.rs b/crates/openshell-cli/src/commands/provider.rs index 5ec34cc316..c89b977b96 100644 --- a/crates/openshell-cli/src/commands/provider.rs +++ b/crates/openshell-cli/src/commands/provider.rs @@ -1944,7 +1944,7 @@ pub async fn provider_refresh_status( fn refresh_status_header() -> String { format!( - "{:<24} {:<28} {:<28} {:<24} {:<18} {:<20} {:<20} {:<20} {:<44} {}", + "{:<24} {:<28} {:<28} {:<24} {:<18} {:<20} {:<20} {:<20} {:<44} {:<32} {}", "PROVIDER".bold(), "CREDENTIAL_KEY".bold(), "STRATEGY".bold(), @@ -1954,6 +1954,7 @@ fn refresh_status_header() -> String { "NEXT_REFRESH".bold(), "LAST_REFRESH".bold(), "FAILURE_CODE".bold(), + "MATERIAL_SECRETS".bold(), "LAST_ERROR".bold(), ) } @@ -2169,7 +2170,7 @@ fn refresh_status_row(status: &ProviderCredentialRefreshStatus) -> String { let recovery_action = ProviderCredentialRefreshRecoveryAction::try_from(status.recovery_action) .unwrap_or(ProviderCredentialRefreshRecoveryAction::Unspecified); format!( - "{:<24} {:<28} {:<28} {:<24} {:<18} {:<20} {:<20} {:<20} {:<44} {}", + "{:<24} {:<28} {:<28} {:<24} {:<18} {:<20} {:<20} {:<20} {:<44} {:<32} {}", status.provider, status.credential_key, provider_refresh_strategy_name(strategy), @@ -2182,10 +2183,25 @@ fn refresh_status_row(status: &ProviderCredentialRefreshStatus) -> String { ), format_optional_epoch_ms(proto_timestamp_ms(status.last_refresh_time.as_ref())), status.failure_code, + material_secrets_field(&status.material_secrets), truncate_status_field(&status.last_error, 72), ) } +/// Material keys supplied by workspace secrets, as `key=secret`, sorted. These +/// are the credentials to convert with `provider refresh inline-secrets`. +fn material_secrets_field(material_secrets: &HashMap) -> String { + if material_secrets.is_empty() { + return "-".to_string(); + } + let mut bindings: Vec<_> = material_secrets + .iter() + .map(|(key, secret)| format!("{key}={secret}")) + .collect(); + bindings.sort(); + bindings.join(",") +} + pub fn provider_refresh_recovery_action_name( action: ProviderCredentialRefreshRecoveryAction, ) -> &'static str { @@ -3509,6 +3525,7 @@ binaries: [/usr/bin/curl] assert!(header.contains("RECOVERY")); assert!(header.contains("FAILURE_CODE")); assert!(header.contains("LAST_ERROR")); + assert!(header.contains("MATERIAL_SECRETS")); let row = refresh_status_row(&ProviderCredentialRefreshStatus { provider: "my-graph".to_string(), @@ -3537,6 +3554,21 @@ binaries: [/usr/bin/curl] assert!(row.contains("2026-01-01 00:00:00")); assert!(!row.contains("292278994")); assert!(row.contains("...")); + assert!(row.contains(" - "), "no secret references render as '-'"); + } + + #[test] + fn refresh_status_row_lists_workspace_secret_bindings() { + let row = refresh_status_row(&ProviderCredentialRefreshStatus { + provider: "alice-mcp".to_string(), + credential_key: "MCP_ACCESS_TOKEN".to_string(), + material_secrets: HashMap::from([ + ("private_key".to_string(), "signing-key".to_string()), + ("client_secret".to_string(), "acme-oauth-client".to_string()), + ]), + ..Default::default() + }); + assert!(row.contains("client_secret=acme-oauth-client,private_key=signing-key")); } #[test] diff --git a/crates/openshell-server/src/grpc/provider.rs b/crates/openshell-server/src/grpc/provider.rs index aad8c606ca..27afe1ca53 100644 --- a/crates/openshell-server/src/grpc/provider.rs +++ b/crates/openshell-server/src/grpc/provider.rs @@ -3622,6 +3622,18 @@ fn validate_material_secret_references( Ok(()) } +/// Material keys read when refresh is configured, not when it mints. +const CONFIGURATION_MATERIAL_KEYS: &[&str] = &[ + "token_url", + "token_uri", + "sts_endpoint_url", + "tenant_id", + "scope", + "scopes", + "refresh_before_seconds", + "max_lifetime_seconds", +]; + /// Only secret minting material may come from a shared workspace secret, and /// never material the minter writes back for one provider. fn validate_material_secret_keys( @@ -3630,6 +3642,13 @@ fn validate_material_secret_keys( material_secrets: &HashMap, ) -> Result<(), Status> { for key in material_secrets.keys() { + // Configure reads these from inline material only, so a reference would + // be recorded but silently ignored, even if a profile marks it secret. + if CONFIGURATION_MATERIAL_KEYS.contains(&key.as_str()) { + return Err(Status::invalid_argument(format!( + "material key '{key}' configures the refresh and cannot come from a workspace secret" + ))); + } if minter::strategy_rotated_material_keys(strategy).contains(&key.as_str()) { return Err(Status::invalid_argument(format!( "material key '{key}' is rotated per provider and cannot come from a workspace secret" @@ -5602,6 +5621,18 @@ pub(super) async fn handle_inline_provider_refresh_secrets( "provider refresh is being deleted", )); } + // A mint persists this status before contacting the token endpoint and + // stages a rotated refresh token only if its version still matches. + // Replacing the state now would make it discard that token. A mint that + // starts after this read bumps the version, so the write below fails. + if matches!( + refresh.status.as_str(), + "refresh_in_progress" | "refresh_committing" + ) { + return Err(Status::aborted( + "a credential mint is in progress; retry once it finishes, or reconfigure the refresh if its outcome is uncertain", + )); + } let bindings = secret_refs::material_secret_bindings(refresh.metadata.as_ref())?; if bindings.is_empty() { replay_facts.refresh(&refresh)?; diff --git a/crates/openshell-server/src/grpc/secret.rs b/crates/openshell-server/src/grpc/secret.rs index 7d7ee78e57..09f4f763df 100644 --- a/crates/openshell-server/src/grpc/secret.rs +++ b/crates/openshell-server/src/grpc/secret.rs @@ -35,7 +35,8 @@ use crate::auth::workspace_authz::{ use crate::compute::MutationScope; use crate::pagination::Pagination; use crate::persistence::{ - ObjectListQuery, ObjectType, PersistenceError, Store, WriteCondition, current_time_ms, + ObjectCursor, ObjectListQuery, ObjectType, PersistenceError, Store, WriteCondition, + current_time_ms, }; use crate::storage_proto::StoredWorkspaceSecret; @@ -45,7 +46,7 @@ const MAX_SECRETS_PER_WORKSPACE: u32 = 500; const MAX_ALLOWED_ENDPOINTS: usize = 16; const MAX_ENDPOINT_LEN: usize = 2048; const MAX_DESCRIPTION_LEN: usize = 1024; -const SWEEP_LIMIT: u32 = 1000; +const SWEEP_PAGE_SIZE: u32 = 1000; impl ObjectType for StoredWorkspaceSecret { fn object_type() -> &'static str { @@ -280,6 +281,7 @@ pub(super) async fn handle_create_secret( let allowed_endpoints = normalize_allowed_endpoints(&req.allowed_endpoints)?; ensure_workspace_secrets_active(state).await?; ensure_credential_storage(state)?; + let workspace_id = active_workspace_id(&state.store, &workspace).await?; let id = uuid::Uuid::new_v4().to_string(); let now = openshell_core::time::timestamp_from_millis(current_time_ms()).ok(); @@ -331,6 +333,30 @@ pub(super) async fn handle_create_secret( return Err(Status::internal(format!("persist secret failed: {err}"))); } }; + // Workspace deletion marks the workspace terminating before it scans for + // blocking resources. If that scan ran before this insert, the workspace + // is no longer the active one we resolved, so undo the insert rather than + // leave a secret in a deleted workspace whose name can be reused. + let still_active = active_workspace_id(&state.store, &workspace) + .await + .is_ok_and(|current| current == workspace_id); + if !still_active { + if let Err(err) = state + .store + .delete_if( + StoredWorkspaceSecret::object_type(), + &id, + write.resource_version, + ) + .await + { + warn!(secret_id = %id, error = %err, "failed to remove secret created during workspace deletion"); + } + discard_staged_value(state, &workspace, &id, handle).await; + return Err(Status::failed_precondition(format!( + "workspace '{workspace}' was deleted while the secret was being created" + ))); + } if let Some(metadata) = stored.metadata.as_mut() { metadata.resource_version = write.resource_version; } @@ -339,6 +365,21 @@ pub(super) async fn handle_create_secret( })) } +/// ID of the named workspace while it is active. +async fn active_workspace_id(store: &Store, workspace: &str) -> Result { + let record: Option = store + .get_message_by_name("", workspace) + .await + .map_err(|e| Status::internal(format!("fetch workspace failed: {e}")))?; + record + .and_then(|record| record.metadata) + .filter(|metadata| metadata.deletion_time.is_none()) + .map(|metadata| metadata.id) + .ok_or_else(|| { + Status::failed_precondition(format!("workspace '{workspace}' is being deleted")) + }) +} + pub(super) async fn handle_get_secret( state: &Arc, request: Request, @@ -785,11 +826,33 @@ async fn drain_pending_deletions( /// Retry backend deletions and tombstone removal that an earlier request could /// not finish. Runs on the gateway's periodic maintenance tick. pub async fn sweep_workspace_secrets(state: &ServerState) -> Result { - let records = state - .store - .list_by_type(StoredWorkspaceSecret::object_type(), SWEEP_LIMIT, 0) - .await - .map_err(|e| Status::internal(format!("list secrets failed: {e}")))?; + let mut finished = 0; + let mut after: Option = None; + // Page through every record: healthy secrets never leave the listing, so + // rereading one fixed page would starve later pending cleanup. + loop { + let records = state + .store + .list_by_type_after( + StoredWorkspaceSecret::object_type(), + after.as_ref(), + SWEEP_PAGE_SIZE, + ) + .await + .map_err(|e| Status::internal(format!("list secrets failed: {e}")))?; + let last_page = records.len() < SWEEP_PAGE_SIZE as usize; + after = records.last().map(ObjectCursor::from); + finished += sweep_page(state, records).await?; + if last_page { + return Ok(finished); + } + } +} + +async fn sweep_page( + state: &ServerState, + records: Vec, +) -> Result { let mut finished = 0; for record in records { let mut stored = StoredWorkspaceSecret::decode(record.payload.as_slice()) @@ -1208,4 +1271,85 @@ mod tests { }); assert_eq!(reveal(&state, operator).await.unwrap(), "the-value"); } + + #[tokio::test] + async fn a_create_racing_workspace_deletion_leaves_nothing_behind() { + use openshell_core::proto::{CreateWorkspaceRequest, DeleteWorkspaceRequest}; + let state = enabled_state().await; + super::super::workspace::handle_create_workspace( + &state, + Request::new(CreateWorkspaceRequest { + request_id: String::new(), + name: "team".to_string(), + labels: HashMap::new(), + }), + ) + .await + .unwrap(); + let baseline = state.credentials.stored_credential_count(); + + // Pause the create after its activity check, while it stages the value. + let (entered, release) = state.credentials.gate_next_store(); + let creator = Arc::clone(&state); + let create = tokio::spawn(async move { + let mut request = create_request("acme", "value"); + request.workspace_scope = Some(workspace_selector("team".to_string())); + handle_create_secret(&creator, authed_request(request)).await + }); + entered.await.unwrap(); + // The scan finds no secret yet, so the deletion completes. + super::super::workspace::handle_delete_workspace( + &state, + Request::new(DeleteWorkspaceRequest { + request_id: String::new(), + allow_missing: false, + name: "team".to_string(), + }), + ) + .await + .expect("workspace deletion completes"); + release.send(()).unwrap(); + + let err = create.await.unwrap().unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + assert!( + state + .store + .get_message_by_name::("team", "acme") + .await + .unwrap() + .is_none(), + "no secret row outlives the workspace" + ); + assert_eq!( + state.credentials.stored_credential_count(), + baseline, + "the staged value is discarded" + ); + } + + #[tokio::test] + async fn sweep_reaches_pending_cleanup_behind_a_full_page_of_healthy_secrets() { + let state = enabled_state().await; + for index in 0..SWEEP_PAGE_SIZE { + let healthy = StoredWorkspaceSecret { + metadata: Some(ObjectMeta { + id: format!("healthy-{index:04}"), + name: format!("healthy-{index:04}"), + // Another workspace, so the pending secret stays under + // the per-workspace quota. The sweep covers every workspace. + workspace: "elsewhere".to_string(), + ..ObjectMeta::default() + }), + ..Default::default() + }; + state.store.put_message(&healthy).await.unwrap(); + } + create(&state, "zzz-pending", "value").await; + let record = stored(&state, "zzz-pending").await.unwrap(); + tombstone(&state, record).await.unwrap(); + + assert_eq!(sweep_workspace_secrets(&state).await.unwrap(), 1); + assert!(stored(&state, "zzz-pending").await.is_none()); + } } diff --git a/crates/openshell-server/src/grpc/secret_refresh_tests.rs b/crates/openshell-server/src/grpc/secret_refresh_tests.rs index 2cf22fa82e..ed6f32d27d 100644 --- a/crates/openshell-server/src/grpc/secret_refresh_tests.rs +++ b/crates/openshell-server/src/grpc/secret_refresh_tests.rs @@ -62,6 +62,15 @@ async fn import_profile( state: &Arc, strategy: ProviderCredentialRefreshStrategy, token_url: &str, +) { + import_profile_with_material(state, strategy, token_url, Vec::new()).await; +} + +async fn import_profile_with_material( + state: &Arc, + strategy: ProviderCredentialRefreshStrategy, + token_url: &str, + extra_material: Vec, ) { let credential = ProviderProfileCredential { token_grant_owners: Vec::new(), @@ -87,10 +96,13 @@ async fn import_profile( nanos: 0, }), additional_outputs: Vec::new(), - material: vec![ + material: [ material("client_id", false), material("client_secret", true), - ], + ] + .into_iter() + .chain(extra_material) + .collect(), }), token_grant: None, }; @@ -509,3 +521,128 @@ async fn a_refresh_whose_secret_disappeared_fails_closed() { let stored = refresh_state(&state).await; assert_eq!(stored.failure_code, "referenced_secret_missing"); } + +#[tokio::test] +async fn configure_time_keys_cannot_come_from_a_secret_even_when_marked_secret() { + let state = enabled_state().await; + let token_url = "https://login.example.com/token"; + import_profile_with_material( + &state, + ProviderCredentialRefreshStrategy::Oauth2ClientCredentials, + token_url, + vec![material("refresh_before_seconds", true)], + ) + .await; + create_secret(&state, "600", token_url).await; + let mut request = configure_request(ProviderCredentialRefreshStrategy::Oauth2ClientCredentials); + request + .material + .insert("client_secret".to_string(), "inline".to_string()); + request.material_secrets = + HashMap::from([("refresh_before_seconds".to_string(), "acme".to_string())]); + request.secret_material_keys = vec!["refresh_before_seconds".to_string()]; + let err = configure(&state, request).await.unwrap_err(); + assert_eq!(err.code(), Code::InvalidArgument); + assert!( + err.message().contains("configures the refresh"), + "{}", + err.message() + ); +} + +#[tokio::test] +async fn inlining_never_overlaps_a_rotating_mint() { + let server = MockServer::start().await; + Mock::given(method("POST")) + .and(path("/token")) + .and(body_string_contains("refresh_token=original-refresh")) + .respond_with( + ResponseTemplate::new(200) + .set_body_json(serde_json::json!({ + "access_token": "token-1", + "refresh_token": "rotated-refresh", + "expires_in": 3600 + })) + .set_delay(std::time::Duration::from_millis(500)), + ) + .expect(1) + .mount(&server) + .await; + let token_url = format!("{}/token", server.uri()); + let state = enabled_state().await; + import_profile( + &state, + ProviderCredentialRefreshStrategy::Oauth2RefreshToken, + &token_url, + ) + .await; + create_secret(&state, "client-secret", &token_url).await; + let mut request = configure_request(ProviderCredentialRefreshStrategy::Oauth2RefreshToken); + request + .material + .insert("refresh_token".to_string(), "original-refresh".to_string()); + configure(&state, request).await.expect("configure"); + + let minter = Arc::clone(&state); + let mint = tokio::spawn(async move { rotate(&minter).await }); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(5); + while refresh_state(&state).await.status != "refresh_in_progress" { + assert!(std::time::Instant::now() < deadline, "mint never started"); + tokio::time::sleep(std::time::Duration::from_millis(10)).await; + } + + let err = inline(&state).await.unwrap_err(); + assert_eq!(err.code(), Code::Aborted); + + mint.await + .unwrap() + .expect("the mint keeps its rotated refresh token"); + assert_eq!( + stored_refresh_token(&state).await.as_deref(), + Some("rotated-refresh") + ); + + // Once the mint finishes, inlining keeps the rotated token. + inline(&state).await.expect("inline after the mint"); + let stored = refresh_state(&state).await; + assert!( + secret_refs::material_secret_bindings(stored.metadata.as_ref()) + .unwrap() + .is_empty() + ); + assert_eq!( + stored_refresh_token(&state).await.as_deref(), + Some("rotated-refresh") + ); +} + +async fn inline( + state: &Arc, +) -> Result< + tonic::Response, + tonic::Status, +> { + handle_inline_provider_refresh_secrets( + state, + authed_request(InlineProviderRefreshSecretsRequest { + workspace_scope: Some(default_workspace()), + provider: PROVIDER.to_string(), + credential_key: KEY.to_string(), + request_id: String::new(), + }), + ) + .await +} + +async fn stored_refresh_token(state: &Arc) -> Option { + let stored = refresh_state(state).await; + state + .credentials + .resolve_refresh_material( + openshell_credential_issuer::refresh::refresh_material_scope(&stored), + &stored.secret_material_handles, + ) + .await + .unwrap() + .remove("refresh_token") +} From 739e763064167f4bef9f442e04011fd7f4fd55fe Mon Sep 17 00:00:00 2001 From: Mrunal Patel Date: Wed, 18 Nov 2026 15:28:57 +0000 Subject: [PATCH 08/10] docs: clarify workspace secret rollback Point operators at the MATERIAL_SECRETS status column, note that inlining waits for an in-progress mint, and describe how an older gateway mints copied values after a downgrade. Update the RFC to match. Refs #5563 Signed-off-by: Mrunal Patel --- docs/how-it-works/gateways/configuration.mdx | 10 ++++++++-- rfc/NNNN-workspace-secrets/README.md | 18 ++++++++++++++---- 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/docs/how-it-works/gateways/configuration.mdx b/docs/how-it-works/gateways/configuration.mdx index 200815c6d3..3a6ff710a8 100644 --- a/docs/how-it-works/gateways/configuration.mdx +++ b/docs/how-it-works/gateways/configuration.mdx @@ -706,10 +706,16 @@ flag attests that they are retired. Before rolling back to a release without workspace secrets: 1. Set `enable_workspace_secrets = false`. -2. For each credential whose `provider refresh status` lists a secret, run - `openshell provider refresh inline-secrets`. +2. For each row of `openshell provider refresh status` with a value in the + `MATERIAL_SECRETS` column, run `openshell provider refresh inline-secrets` + for that provider and credential key. Retry it if a mint is in progress. 3. Delete every workspace secret. +After the downgrade, the older gateway mints copied values without the +allowed-endpoint and redirect checks. It still sends them only to the token +endpoint fixed when refresh was configured, but it follows redirects from that +endpoint. + `[openshell.gateway] enable_secret_reveal` defaults to `false`. When `true`, the `RevealSecret` RPC returns secret values to operators authenticated with operator mTLS, and to OIDC workspace admins whose token carries the literal diff --git a/rfc/NNNN-workspace-secrets/README.md b/rfc/NNNN-workspace-secrets/README.md index 71a7b89908..1d51aaaedc 100644 --- a/rfc/NNNN-workspace-secrets/README.md +++ b/rfc/NNNN-workspace-secrets/README.md @@ -207,8 +207,10 @@ records the superseded handle as a pending deletion and then deletes it. A failed swap deletes the staged handle. `DeleteSecret` tombstones the record by setting its deletion time, deletes the current and pending handles, then removes the row. The periodic refresh worker tick also sweeps workspace-secret -tombstones and pending deletions, so a backend failure is retried rather than -leaking a handle. The next `UpdateSecret` or `DeleteSecret` also retries them. +tombstones and pending deletions, paging through every record so healthy +secrets cannot starve later cleanup. A backend failure is therefore retried +rather than leaking a handle. The next `UpdateSecret` or `DeleteSecret` also +retries them. The CLI adds `openshell secret create|update|get|list|delete`. Values come from `--value-env VAR` or `--value-file PATH`, never from a command-line argument. @@ -386,7 +388,10 @@ conflicts with `ConfigureProviderRefresh`, so a new reference cannot appear between the check and the delete. Deleting a provider or its refresh configuration removes its references. Workspace deletion is blocked while secrets exist; the secret type joins the resource kinds that workspace deletion -checks. +checks. Workspace deletion marks the workspace terminating before it scans for +blockers, so `CreateSecret` re-reads the workspace after its insert. If the +workspace is no longer the active one it resolved, the create removes its row +and staged value and fails. Otherwise the deletion's scan finds the secret. Bindings use immutable IDs. Deleting a secret and recreating one with the same name therefore never silently rebinds a provider. @@ -483,7 +488,12 @@ support, the operator must: 3. Delete all secrets, which the delete guard now permits. The rollback documentation states this. `ListSecrets` in each workspace, plus -refresh status, shows what remains. Rolling back without these steps is +the referenced secrets that refresh status reports for each credential, shows +what remains. Inlining refuses while a mint is in progress, because replacing +the refresh state then would make the mint discard a rotated refresh token. +After the downgrade, the older minter applies its legacy behavior to the copied +values: no endpoint constraint and default redirect handling, toward the token +endpoint fixed at configuration. Rolling back without these steps is unsupported. In that case the protections below keep older replicas from minting with missing material. They do not protect secrets themselves: an older replica can still delete a workspace that holds secrets, as described From 9146f998b83f6f5f90accddc5f0fdcda35b0ce6a Mon Sep 17 00:00:00 2001 From: Mrunal Patel Date: Wed, 18 Nov 2026 15:40:54 +0000 Subject: [PATCH 09/10] fix(server): address second workspace secret review - Keep secrets whose deletion did not finish visible in Get and List, with their deletion time set, so list pages stay full. Reject updates to them, and let CreateSecret finish an interrupted deletion of the same name instead of failing with already exists. The CLI shows them as deleting. - Check the RevealSecret principal and workspace role before the enable_secret_reveal setting, so unauthorized callers cannot probe it. - Record provider and credential targets in the InlineProviderRefreshSecrets audit event. - Re-read a referenced secret once when inlining races an update, and queue any handle a copied value replaces for deletion. Refs #5563 Signed-off-by: Mrunal Patel --- crates/openshell-cli/src/commands/secret.rs | 43 +++++- crates/openshell-server/src/grpc/mod.rs | 12 +- .../src/grpc/mutation_audit.rs | 2 + crates/openshell-server/src/grpc/provider.rs | 115 +++++++++------ crates/openshell-server/src/grpc/secret.rs | 139 +++++++++++++++--- 5 files changed, 247 insertions(+), 64 deletions(-) diff --git a/crates/openshell-cli/src/commands/secret.rs b/crates/openshell-cli/src/commands/secret.rs index 8dbdd0cc6c..c1ecf59bf5 100644 --- a/crates/openshell-cli/src/commands/secret.rs +++ b/crates/openshell-cli/src/commands/secret.rs @@ -189,6 +189,13 @@ pub async fn secret_get( secret.allowed_endpoints.join(", ") ); println!(" {} {}", "Value updated:".dimmed(), value_updated(&secret)); + if is_deleting(&secret) { + println!( + " {} deletion did not finish; rerun `openshell secret delete {}`", + "Deleting:".dimmed(), + secret_name(&secret) + ); + } if let Some(meta) = &secret.metadata && !meta.labels.is_empty() { @@ -245,15 +252,17 @@ pub async fn secret_list( .unwrap_or(4) .max(4); println!( - "{: &str { .map_or("", |meta| meta.name.as_str()) } +fn is_deleting(secret: &Secret) -> bool { + secret + .metadata + .as_ref() + .is_some_and(|meta| meta.deletion_time.is_some()) +} + +fn status(secret: &Secret) -> &'static str { + if is_deleting(secret) { + "deleting" + } else { + "active" + } +} + fn value_updated(secret: &Secret) -> String { secret .value_update_time @@ -313,6 +337,7 @@ fn secret_to_json(secret: &Secret) -> serde_json::Value { "labels": meta.labels, "description": secret.description, "allowed_endpoints": secret.allowed_endpoints, + "status": status(secret), "value_updated_at": secret .value_update_time .as_ref() @@ -351,5 +376,19 @@ mod tests { fn json_output_never_carries_a_value_field() { let json = secret_to_json(&Secret::default()); assert!(json.get("value").is_none()); + assert_eq!(json["status"], "active"); + } + + #[test] + fn unfinished_deletions_are_reported() { + let secret = Secret { + metadata: Some(openshell_core::proto::datamodel::v1::ObjectMeta { + deletion_time: Some(prost_types::Timestamp::default()), + ..Default::default() + }), + ..Default::default() + }; + assert_eq!(status(&secret), "deleting"); + assert_eq!(secret_to_json(&secret)["status"], "deleting"); } } diff --git a/crates/openshell-server/src/grpc/mod.rs b/crates/openshell-server/src/grpc/mod.rs index 6275d85efb..4f928633b1 100644 --- a/crates/openshell-server/src/grpc/mod.rs +++ b/crates/openshell-server/src/grpc/mod.rs @@ -789,7 +789,17 @@ impl OpenShell for OpenShellService { &self, request: Request, ) -> Result, Status> { - mutation_replay::run(&self.state, request).await + let targets = { + let request = request.get_ref(); + mutation_audit::scoped( + request.workspace_scope.as_ref(), + [ + mutation_audit::requested("provider", &request.provider), + mutation_audit::requested("provider_credential", &request.credential_key), + ], + ) + }; + mutation_audit::audited(targets, mutation_replay::run(&self.state, request)).await } async fn delete_provider_refresh( diff --git a/crates/openshell-server/src/grpc/mutation_audit.rs b/crates/openshell-server/src/grpc/mutation_audit.rs index 44f27a7de3..62a9ec280f 100644 --- a/crates/openshell-server/src/grpc/mutation_audit.rs +++ b/crates/openshell-server/src/grpc/mutation_audit.rs @@ -134,6 +134,8 @@ impl Audited for SandboxTemplateResponse { /// the response carries the value and contributes nothing to the record. impl Audited for openshell_core::proto::RevealSecretResponse {} +impl Audited for openshell_core::proto::InlineProviderRefreshSecretsResponse {} + impl Audited for SecretResponse { fn produced(&self) -> Vec> { stored( diff --git a/crates/openshell-server/src/grpc/provider.rs b/crates/openshell-server/src/grpc/provider.rs index 27afe1ca53..16213e8170 100644 --- a/crates/openshell-server/src/grpc/provider.rs +++ b/crates/openshell-server/src/grpc/provider.rs @@ -5643,47 +5643,9 @@ pub(super) async fn handle_inline_provider_refresh_secrets( let mut constraints = secret_refs::material_endpoint_constraints(refresh.metadata.as_ref())?; let mut values = HashMap::new(); for (material_key, binding) in &bindings { - let secret = state - .store - .get_message::(&binding.id) - .await - .map_err(|e| Status::internal(format!("fetch workspace secret failed: {e}")))? - .filter(|secret| { - secret.object_workspace() == workspace - && secret - .metadata - .as_ref() - .is_some_and(|metadata| metadata.deletion_time.is_none()) - }); - let Some((secret, handle)) = secret.and_then(|secret| { - let handle = secret.value_handle.clone()?; - Some((secret, handle)) - }) else { - return Err(Status::failed_precondition(format!( - "workspace secret '{}' no longer exists; reconfigure the refresh grant", - binding.name - ))); - }; - let resolved = state - .credentials - .resolve_secrets( - openshell_credential_issuer::SecretOwner::workspace_secret(&workspace, &binding.id), - &HashMap::from([(secret_refs::WORKSPACE_SECRET_VALUE_KEY.to_string(), handle)]), - 0, - ) - .await?; - let value = resolved - .values - .get(secret_refs::WORKSPACE_SECRET_VALUE_KEY) - .cloned() - .ok_or_else(|| { - Status::unavailable(format!( - "workspace secret '{}' could not be read", - binding.name - )) - })?; + let (value, allowed_endpoints) = read_workspace_secret(state, &workspace, binding).await?; values.insert(material_key.clone(), value); - constraints.insert(material_key.clone(), secret.allowed_endpoints); + constraints.insert(material_key.clone(), allowed_endpoints); } let staging_id = format!( @@ -5700,7 +5662,18 @@ pub(super) async fn handle_inline_provider_refresh_secrets( &HashMap::new(), ) .await?; - refresh.secret_material_handles.extend(staged.clone()); + for (material_key, handle) in &staged { + if let Some(previous) = refresh + .secret_material_handles + .insert(material_key.clone(), handle.clone()) + { + crate::provider_refresh::enqueue_pending_secret_deletion( + &mut refresh, + material_key, + previous, + ); + } + } let expected_version = { let metadata = refresh.metadata.get_or_insert_with(Default::default); secret_refs::set_material_secret_bindings( @@ -5741,6 +5714,66 @@ pub(super) async fn handle_inline_provider_refresh_secrets( })) } +/// Read a referenced workspace secret's value and endpoint constraint. A +/// concurrent `UpdateSecret` can delete the handle read from the record, so a +/// failed read re-reads the record once, as minting does. +async fn read_workspace_secret( + state: &ServerState, + workspace: &str, + binding: &secret_refs::MaterialSecretBinding, +) -> Result<(String, Vec), Status> { + let mut last_error = None; + for _ in 0..2 { + let secret = state + .store + .get_message::(&binding.id) + .await + .map_err(|e| Status::internal(format!("fetch workspace secret failed: {e}")))? + .filter(|secret| { + secret.object_workspace() == workspace + && secret + .metadata + .as_ref() + .is_some_and(|metadata| metadata.deletion_time.is_none()) + }); + let Some((secret, handle)) = secret.and_then(|secret| { + let handle = secret.value_handle.clone()?; + Some((secret, handle)) + }) else { + return Err(Status::failed_precondition(format!( + "workspace secret '{}' no longer exists; reconfigure the refresh grant", + binding.name + ))); + }; + match state + .credentials + .resolve_secrets( + openshell_credential_issuer::SecretOwner::workspace_secret(workspace, &binding.id), + &HashMap::from([(secret_refs::WORKSPACE_SECRET_VALUE_KEY.to_string(), handle)]), + 0, + ) + .await + { + Ok(mut resolved) => { + if let Some(value) = resolved + .values + .remove(secret_refs::WORKSPACE_SECRET_VALUE_KEY) + .filter(|value| !value.is_empty()) + { + return Ok((value, secret.allowed_endpoints)); + } + last_error = Some("credential storage returned no value".to_string()); + } + Err(err) => last_error = Some(err.message().to_string()), + } + } + Err(Status::unavailable(format!( + "workspace secret '{}' could not be read: {}", + binding.name, + last_error.unwrap_or_default() + ))) +} + /// Remove the expiry for each key the refresh still owns — one whose stored /// value equals the refresh's `expires_at_ms`. Independently updated expiries (no /// longer matching) are left in place. diff --git a/crates/openshell-server/src/grpc/secret.rs b/crates/openshell-server/src/grpc/secret.rs index 09f4f763df..170d0ccc7f 100644 --- a/crates/openshell-server/src/grpc/secret.rs +++ b/crates/openshell-server/src/grpc/secret.rs @@ -282,6 +282,7 @@ pub(super) async fn handle_create_secret( ensure_workspace_secrets_active(state).await?; ensure_credential_storage(state)?; let workspace_id = active_workspace_id(&state.store, &workspace).await?; + clear_interrupted_deletion(state, &workspace, &req.name).await?; let id = uuid::Uuid::new_v4().to_string(); let now = openshell_core::time::timestamp_from_millis(current_time_ms()).ok(); @@ -365,6 +366,29 @@ pub(super) async fn handle_create_secret( })) } +/// Finish a deletion of `name` that an earlier request could not complete, so +/// its name can be reused. Fails when the earlier deletion still cannot finish. +async fn clear_interrupted_deletion( + state: &ServerState, + workspace: &str, + name: &str, +) -> Result<(), Status> { + let existing = state + .store + .get_message_by_name::(workspace, name) + .await + .map_err(|e| Status::internal(format!("fetch secret failed: {e}")))?; + let Some(existing) = existing.filter(is_tombstoned) else { + return Ok(()); + }; + finish_deletion(state, existing).await.map_err(|err| { + Status::failed_precondition(format!( + "secret '{name}' is still being deleted and cleanup failed ({}); retry later", + err.message() + )) + }) +} + /// ID of the named workspace while it is active. async fn active_workspace_id(store: &Store, workspace: &str) -> Result { let record: Option = store @@ -397,8 +421,13 @@ pub(super) async fn handle_get_secret( ) .await? .name; - let stored = get_live_secret(&state.store, &workspace, &req.name) - .await? + // A secret whose deletion is unfinished is returned with its deletion + // time set, so it is visible while it still holds its name and quota. + let stored = state + .store + .get_message_by_name::(&workspace, &req.name) + .await + .map_err(|e| Status::internal(format!("fetch secret failed: {e}")))? .ok_or_else(|| Status::not_found("secret not found"))?; Ok(Response::new(SecretResponse { secret: Some(public_secret(&stored)), @@ -444,12 +473,9 @@ pub(super) async fn handle_list_secrets( .map_err(|e| Status::internal(format!("list secrets failed: {e}")))?; let next_page_token = pagination.next_object_token(page.next_cursor.as_ref()); Ok(Response::new(ListSecretsResponse { - secrets: page - .messages - .iter() - .filter(|stored| !is_tombstoned(stored)) - .map(public_secret) - .collect(), + // Unfinished deletions stay in the listing, with their deletion time + // set, so pages stay full and the secrets stay visible. + secrets: page.messages.iter().map(public_secret).collect(), next_page_token, })) } @@ -481,9 +507,18 @@ pub(super) async fn handle_update_secret( validate_labels(&labels.labels)?; } - let current = get_live_secret(&state.store, &workspace, &req.name) - .await? + let current = state + .store + .get_message_by_name::(&workspace, &req.name) + .await + .map_err(|e| Status::internal(format!("fetch secret failed: {e}")))? .ok_or_else(|| Status::not_found("secret not found"))?; + if is_tombstoned(¤t) { + return Err(Status::failed_precondition(format!( + "secret '{}' is being deleted", + req.name + ))); + } let current_version = current.get_resource_version(); if let Some(expected) = req.expected_resource_version && expected != current_version @@ -658,15 +693,9 @@ pub(super) async fn handle_reveal_secret( ) -> Result, Status> { let principal = super::extract_principal(&request)?; let req = request.into_inner(); - if !state.config.workspace_secrets.reveal_enabled { - return Err(Status::failed_precondition( - "secret reveal is not enabled; set enable_secret_reveal for control-plane callers", - )); - } + // Authorize before revealing anything about gateway configuration, so a + // caller who may not reveal cannot learn whether reveal is enabled. ensure_reveal_principal(state, &principal)?; - if req.name.is_empty() { - return Err(Status::invalid_argument("name is required")); - } let workspace = authorized_workspace( state, &principal, @@ -675,6 +704,14 @@ pub(super) async fn handle_reveal_secret( ) .await? .name; + if !state.config.workspace_secrets.reveal_enabled { + return Err(Status::failed_precondition( + "secret reveal is not enabled; set enable_secret_reveal for control-plane callers", + )); + } + if req.name.is_empty() { + return Err(Status::invalid_argument("name is required")); + } let stored = get_live_secret(&state.store, &workspace, &req.name) .await? .ok_or_else(|| Status::not_found("secret not found"))?; @@ -1212,12 +1249,21 @@ mod tests { #[tokio::test] async fn reveal_is_disabled_by_default() { - let state = enabled_state().await; - create(&state, "acme", "the-value").await; + let mut state = reveal_state(Some("scope")).await; + Arc::get_mut(&mut state) + .expect("test state is not shared") + .config + .workspace_secrets + .reveal_enabled = false; let err = reveal(&state, user(IdentityProvider::Oidc, &[REVEAL_SCOPE])) .await .unwrap_err(); assert_eq!(err.code(), Code::FailedPrecondition); + // A caller who may not reveal learns nothing about the setting. + let err = reveal(&state, user(IdentityProvider::Oidc, &["openshell:all"])) + .await + .unwrap_err(); + assert_eq!(err.code(), Code::PermissionDenied); } #[tokio::test] @@ -1352,4 +1398,57 @@ mod tests { assert_eq!(sweep_workspace_secrets(&state).await.unwrap(), 1); assert!(stored(&state, "zzz-pending").await.is_none()); } + + #[tokio::test] + async fn an_unfinished_deletion_stays_visible_and_frees_its_name_on_recreate() { + let state = enabled_state().await; + create(&state, "acme", "value").await; + let record = stored(&state, "acme").await.unwrap(); + tombstone(&state, record).await.unwrap(); + + let fetched = handle_get_secret( + &state, + authed_request(GetSecretRequest { + workspace_scope: Some(workspace_selector("default".to_string())), + name: "acme".to_string(), + }), + ) + .await + .unwrap() + .into_inner() + .secret + .unwrap(); + assert!(fetched.metadata.as_ref().unwrap().deletion_time.is_some()); + let listed = handle_list_secrets( + &state, + authed_request(ListSecretsRequest { + workspace_scope: Some(workspace_selector("default".to_string())), + page_size: 1, + page_token: String::new(), + label_selector: String::new(), + }), + ) + .await + .unwrap() + .into_inner(); + assert_eq!(listed.secrets.len(), 1, "pages are not shortened"); + + let err = handle_update_secret( + &state, + authed_request(UpdateSecretRequest { + workspace_scope: Some(workspace_selector("default".to_string())), + name: "acme".to_string(), + value: Some("next".to_string()), + ..Default::default() + }), + ) + .await + .unwrap_err(); + assert_eq!(err.code(), Code::FailedPrecondition); + assert!(err.message().contains("being deleted")); + + // Recreating the name finishes the earlier deletion first. + let recreated = create(&state, "acme", "fresh").await; + assert!(recreated.metadata.unwrap().deletion_time.is_none()); + } } From 3bcfa18a5a64a61eab835c631031648f990a29d9 Mon Sep 17 00:00:00 2001 From: Mrunal Patel Date: Wed, 18 Nov 2026 15:40:54 +0000 Subject: [PATCH 10/10] docs: describe unfinished secret deletions Refs #5563 Signed-off-by: Mrunal Patel --- docs/how-it-works/providers/overview.mdx | 3 +++ rfc/NNNN-workspace-secrets/README.md | 9 +++++++-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/docs/how-it-works/providers/overview.mdx b/docs/how-it-works/providers/overview.mdx index 51cc53b7a3..1caa633bdb 100644 --- a/docs/how-it-works/providers/overview.mdx +++ b/docs/how-it-works/providers/overview.mdx @@ -370,6 +370,9 @@ These rules apply: `openshell provider refresh inline-secrets` to copy the current value into the provider's own refresh material first. It keeps the current grant, including a rotated refresh token. +- If credential storage fails during deletion, `openshell secret list` shows + the secret as `deleting` until the gateway finishes the cleanup. Rerun + `openshell secret delete` to retry it. `provider refresh status` reports which secrets a credential references. The gateway operator must enable workspace secrets with `enable_workspace_secrets`; diff --git a/rfc/NNNN-workspace-secrets/README.md b/rfc/NNNN-workspace-secrets/README.md index 1d51aaaedc..e2d5c2692b 100644 --- a/rfc/NNNN-workspace-secrets/README.md +++ b/rfc/NNNN-workspace-secrets/README.md @@ -190,7 +190,11 @@ message RevealSecretResponse { ``` Paging and label-selector semantics match `ListProvidersRequest`, except that -`all_workspaces` is rejected. Get and List return metadata only. Values are +`all_workspaces` is rejected. Get and List return metadata only. A secret +whose deletion has not finished is still returned, with `metadata.deletion_time` +set, so listing pages stay full and the secret's name and quota use stay +visible. It cannot be updated, revealed, or referenced, and creating a secret +with the same name first finishes the earlier deletion. Values are bounded by the existing provider map value limit. `allowed_endpoints` cannot be updated. Moving a client to a new token endpoint means creating a new secret, and only someone who knows the value can do that. @@ -348,7 +352,8 @@ Failure classification: | `RevealSecret` | bearer, plus the handler check below | workspace admin | `secret:reveal` | yes, every call | `RevealSecret` fails closed in its handler, whatever the gateway's -authentication mode: +authentication mode. The handler checks the principal and workspace role first, +so a caller who may not reveal cannot learn whether reveal is enabled: - It is disabled unless gateway configuration sets `enable_secret_reveal = true`. Deployments without OCC never expose it.