Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 98 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,104 @@ To remove all resources:
kubectl delete -k openshift
```

## OAuth-protected dev server

`devfile.yaml` runs the Vite dev server behind an [openshift/oauth-proxy](https://github.com/openshift/oauth-proxy)
sidecar, on a dedicated OpenShift Route. Only users who can `get pods` in the workspace
namespace can reach it; the dev server itself is not exposed (`exposure: none`), so it is
reachable only over the pod loopback.

This is meant for internal development on a cluster, not as a general-purpose hardening
recipe. A dedicated Route is needed because both the DevWorkspace basic solver and the Che
gateway put every endpoint on the shared workspace hostname under a rewritten path prefix,
where the `/oauth/callback` path would not route back to the proxy.

### How it is wired

| Piece | Where it lives |
|-------|----------------|
| `oauth-proxy` sidecar, listening on 4180, upstream `127.0.0.1:5173` | `oauth-proxy` container component |
| Service `wizard-oauth` | generated by the operator from the endpoint's `discoverable: true` attribute |
| Route `wizard-oauth` (edge TLS, redirect) | `oauth-proxy-route` OpenShift component, `deployByDefault: true` |
| OAuth client identity | `wizard-oauth` ServiceAccount, created once in the namespace |

The workspace pod keeps the ServiceAccount the DevWorkspace Operator generates for it, so it
keeps its SCC and role bindings. The proxy authenticates as the `wizard-oauth` OAuth client
using that ServiceAccount's token, mounted into the workspace as an automounted secret.

### One-time namespace setup

Run once per namespace, before starting the workspace:

```bash
oc create sa wizard-oauth

# Registers the Route as a valid redirect target, so the callback URL is resolved
# dynamically and never has to be hardcoded.
oc annotate sa wizard-oauth \
serviceaccounts.openshift.io/oauth-redirectreference.primary='{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"wizard-oauth"}}'

# Token for that OAuth client, automounted into the workspace at /var/run/oauth-client.
oc apply -f - <<'EOF'
apiVersion: v1
kind: Secret
metadata:
name: wizard-oauth-client
labels:
controller.devfile.io/mount-to-devworkspace: "true"
controller.devfile.io/watch-secret: "true"
annotations:
kubernetes.io/service-account.name: wizard-oauth
controller.devfile.io/mount-as: file
controller.devfile.io/mount-path: /var/run/oauth-client
type: kubernetes.io/service-account-token
EOF
```

Both labels are required: `mount-to-devworkspace` is what the operator selects on, and
without `watch-secret` the controller's cache never sees the secret. The secret has to exist
before the workspace starts, since automounting happens before inlined OpenShift components
are applied.

Then set the two variables at the top of `devfile.yaml`:

| Variable | Value |
|----------|-------|
| `namespace` | the namespace the workspace runs in — used for `--client-id` and the access check |
| `cookie-secret` | `openssl rand -base64 32` |

Once the workspace is running, the URL is:

```bash
oc get route wizard-oauth -o jsonpath='{.spec.host}{"\n"}'
```

### Notes and troubleshooting

- **`unable to validate against any security context constraint`** — something overrode the
pod's ServiceAccount. The SCC is bound to the generated `<workspaceID>-sa`; a pod running
as any other ServiceAccount loses it.
- **`invalid client`** on login — the `namespace` variable does not match the real namespace,
so `--client-id` names a ServiceAccount that does not exist.
- **403 after logging in** — expected when the user cannot `get pods` in the namespace. Adjust
the `--openshift-sar` argument to match who should have access.
- **502 with an empty page** — that is oauth-proxy failing to reach the upstream, not the
router (router errors come with an HTML page). Either the dev server is not running, so run
the `start-dev-server` command, or it is not on the loopback address the proxy dials. Check
with `oc logs <workspace-pod> -c oauth-proxy`, which logs the dial error. The
`start-dev-server` command passes `--host 127.0.0.1` for this reason: Vite's default host is
`localhost`, which Node may resolve to `::1` only.
- Do not pre-create the `wizard-oauth` Route by hand: the operator refuses to adopt objects it
does not own and the workspace will fail to start.
- The ServiceAccount token is mounted into every workspace container, including `nodejs-dev`.
- Names are fixed, so two workspaces running this devfile in the same namespace collide on the
Service and Route.
- Termination is `edge`, so router-to-pod traffic is plaintext inside the cluster. Reencrypt
would need the serving-cert secret mounted into the sidecar, which devfile container
components cannot do directly.
- To get the usual unauthenticated (or gateway-authenticated) dev server URL back, set the
`dev-server` endpoint to `exposure: public`.

## Technology Stack

- **Frontend Framework:** React 18.3.1
Expand Down
97 changes: 95 additions & 2 deletions devfile.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,22 +16,113 @@ metadata:
website: https://devfile.io
attributes:
controller.devfile.io/storage-type: ephemeral
variables:
# CHANGE_ME - Namespace the workspace runs in. Used for the access check performed by
# oauth-proxy: only users who can `get pods` in this namespace are let through.
namespace: ibuziuk-dev
# CHANGE_ME - Seed for the session cookie: 'openssl rand -base64 32'
cookie-secret: k0uiU7MQGwKY4BqHa22cjOZsmfI2FZzO7SioUUtf4e0=
components:
- name: nodejs-dev
container:
image: quay.io/devfile/universal-developer-image:ubi10-latest
mountSources: true
sourceMapping: /projects
endpoints:
# Not exposed directly: the dev server is only reachable through the
# oauth-proxy sidecar below. Set `exposure: public` to also get the
# usual unauthenticated (or gateway-authenticated) workspace URL back.
- name: dev-server
targetPort: 5173
protocol: http
exposure: public
exposure: none
env:
- name: NODE_ENV
value: development
memoryLimit: 2Gi
cpuLimit: 1000m
# OpenShift OAuth in front of the Vite dev server. The proxy shares the pod
# network with nodejs-dev, so it reaches Vite over 127.0.0.1 and Vite itself
# never has to listen on a routable address.
#
# One-time setup in the workspace namespace (see README section "OAuth-protected
# dev server"). The pod keeps the ServiceAccount the operator generates for it -
# the proxy authenticates as the `wizard-oauth` OAuth client using a token that
# is mounted into the workspace, so no pod ServiceAccount override (and no SCC
# grant) is needed:
#
# oc create sa wizard-oauth
# oc annotate sa wizard-oauth \
# serviceaccounts.openshift.io/oauth-redirectreference.primary='{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"wizard-oauth"}}'
# oc apply -f - <<'EOF'
# apiVersion: v1
# kind: Secret
# metadata:
# name: wizard-oauth-client
# labels:
# controller.devfile.io/mount-to-devworkspace: "true"
# controller.devfile.io/watch-secret: "true"
# annotations:
# kubernetes.io/service-account.name: wizard-oauth
# controller.devfile.io/mount-as: file
# controller.devfile.io/mount-path: /var/run/oauth-client
# type: kubernetes.io/service-account-token
# EOF
- name: oauth-proxy
container:
image: quay.io/openshift/origin-oauth-proxy:4.18
mountSources: false
command:
- /usr/bin/oauth-proxy
args:
- --provider=openshift
- --http-address=0.0.0.0:4180
- --https-address=
- --upstream=http://127.0.0.1:5173
# OAuth client is the wizard-oauth ServiceAccount; its token is automounted
# from the wizard-oauth-client secret created in the one-time setup above.
- --client-id=system:serviceaccount:{{namespace}}:wizard-oauth
- --client-secret-file=/var/run/oauth-client/token
- --openshift-ca=/etc/pki/tls/cert.pem
- --openshift-ca=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
# Access check, run with the end user's token: only users who can get pods
# in this namespace are let through.
- '--openshift-sar={"namespace":"{{namespace}}","resource":"pods","verb":"get"}'
- --cookie-secret={{cookie-secret}}
- --cookie-secure=true
- --pass-basic-auth=false
- --skip-provider-button=true
endpoints:
# `discoverable` makes the operator create a Service named after the
# endpoint, which gives the Route below a stable target to point at.
- name: wizard-oauth
targetPort: 4180
protocol: http
exposure: internal
attributes:
discoverable: true
memoryLimit: 256Mi
memoryRequest: 64Mi
cpuLimit: 200m
cpuRequest: 50m
- name: oauth-proxy-route
openshift:
deployByDefault: true
inlined: |
apiVersion: route.openshift.io/v1
kind: Route
metadata:
name: wizard-oauth
spec:
to:
kind: Service
name: wizard-oauth
weight: 100
port:
targetPort: 4180
tls:
termination: edge
insecureEdgeTerminationPolicy: Redirect
- name: node-modules
volume:
size: 1Gi
Expand All @@ -46,7 +137,9 @@ commands:
- id: start-dev-server
exec:
component: nodejs-dev
commandLine: npm run dev
# Pinned to the IPv4 loopback: vite's default host is `localhost`, which Node
# may resolve to ::1 only, while the oauth-proxy sidecar dials 127.0.0.1.
commandLine: npm run dev -- --host 127.0.0.1
workingDir: ${PROJECT_SOURCE}
group:
kind: run
Expand Down
Loading