Workload is a Helm library chart for Kubernetes workloads. You declare workers, jobs and crons by name in your chart values, and the library renders the Deployments, Jobs and CronJobs that run them, along with their Services, autoscalers, routes and monitors.
- Prerequisites
- Getting started
- What gets rendered
- Configuration details
- Exposing a worker over HTTP
- Mounting config files and secrets
- Keeping secrets out of your values
- Persistence
- Init containers and sidecars
- Running a job as a Helm hook
- Images
- Ports and environment variables
- Scaling
- Rejected combinations
- Service accounts and RBAC
- Resource names and labels
- Templated values
- Empty values and Kubernetes defaults
- Parameters
- Helm 3.8 or later, for OCI dependencies.
- Kubernetes 1.23 or later, tested against 1.37.
- The CRDs of each optional feature you enable:
| Feature | Required CRDs |
|---|---|
externalSecrets |
External Secrets Operator v1 |
httpRoute |
Gateway API v1 |
ingressRoute |
Traefik |
keda |
KEDA |
podMonitor |
Prometheus Operator |
serviceMonitor |
Prometheus Operator |
vpa |
Vertical Pod Autoscaler |
A library chart can't be installed on its own, so you add it as a dependency of your application chart and let a single template render everything.
-
Declare the dependency in your Chart.yaml, then run
helm dependency update ./chart:dependencies: - name: workload version: x.y.z # replace with a released version repository: oci://registry.gitlab.com/kickr-dev
Released versions are listed on Artifact Hub and in the changelog.
-
Create a template, for instance templates/application.yaml, that includes the library:
{{ include "workload.application" . }} -
Declare your components in values.yaml, knowing that workers run continuously, jobs run once and crons run on a schedule. The example below is the smallest working setup, and chart/values.yaml has every key with its comments if you'd rather start from there:
workers: api: kind: Deployment restartPolicy: Always image: registry: registry.example.com repository: api ports: http: 8080 jobs: migrate: restartPolicy: OnFailure image: registry: registry.example.com repository: migrate crons: cleanup: schedule: "0 0 * * *" restartPolicy: OnFailure image: registry: registry.example.com repository: cleanup
Every component needs an image repository and a restart policy, workers a kind as well, and crons a schedule. Their names are DNS-1123 labels of up to 20 characters, unique across all three kinds.
-
Copy chart/values.schema.json next to your values.yaml so that Helm validates your values on install, upgrade, template and lint.
-
Deploy the release:
helm upgrade --install my-release ./chart --namespace my-namespace
Each component gets its controller, and every other resource below appears only for the entries you declare or the features you enable.
| Resource | Rendered for | Rendered when |
|---|---|---|
| ServiceAccount | release, each component | serviceAccount.create set |
| Role and RoleBinding | release, each component | serviceAccount.rules set |
| ConfigMap | each configMaps entry |
always |
| Secret | each secrets entry |
always |
| ExternalSecret | each externalSecrets entry |
always |
| PersistentVolumeClaim | each volumeClaims entry |
not a StatefulSet worker |
| Deployment | each worker | kind: Deployment |
| StatefulSet | each worker | kind: StatefulSet |
| DaemonSet | each worker | kind: DaemonSet |
| Job | each job, per revision | hook unset |
| Job as Helm hook | each job | hook set |
| CronJob | each cron | always |
| Service | each worker | service.enabled and ports set |
| Headless Service | each worker | kind: StatefulSet |
| HorizontalPodAutoscaler | each worker | autoscaling enabled |
| ScaledObject | each worker | keda enabled |
| VerticalPodAutoscaler | each worker | vpa enabled |
| PodDisruptionBudget | each worker | pdb enabled |
| IngressRoute | each worker | ingressRoute enabled |
| HTTPRoute | each worker | httpRoute enabled |
| PodMonitor | each worker | podMonitor enabled |
| ServiceMonitor | each worker | serviceMonitor enabled |
To reach a worker from outside the cluster, enable ingressRoute for Traefik,
or httpRoute with its parent gateways and hostnames for the Gateway API, and point it at one of the worker's ports by name.
Since routes are rendered with tpl, the example below declares the host once and reuses it in each route:
host: api.example.com
workers:
api:
kind: Deployment
restartPolicy: Always
image:
registry: registry.example.com
repository: api
ports:
http: 8080
ingressRoute:
enabled: true
port: http
entryPoints:
- websecure
routes:
- match: Host(`{{ .Values.host }}`) && PathPrefix(`/api`)
- match: Host(`{{ .Values.host }}`) && PathPrefix(`/docs`)
tls:
certResolver: letsencryptEach entry under configMaps or secrets renders a resource named <fullname>-<component>-<name>,
and a worker rolls out whenever one of them changes.
Its content merges any of:
files, chart files matching the globs, copied as they are.templates, chart files matching the globs, rendered with tpl.data, inline entries, rendered with tpl.
The container then reads an entry in one of two ways:
- As files, when
mountPathis set, through a read-only pod volume namedconfigmap-<name>orsecret-<name>that init containers and sidecars can mount too. - As environment variables, through
envFrom, whose names are rendered with tpl.
The example below does both, mounting config files and building the Secret name with the workload.componentName helper:
db:
password: "" # comes from DB_PASSWORD CICD env variable
workers:
api:
configMaps:
- name: config.
mountPath: /etc/config/
files: [config/*]
secrets:
- name: credentials.
data:
DB_PASSWORD: "{{ .Values.db.password }}"
envFrom:
- secretRef:
name: '{{ include "workload.componentName" . }}-credentials'A ConfigMap or Secret from your own chart templates is yours to mount through volumes and volumeMounts.
Volumes are rendered with tpl as well, so the name can follow the release fullname:
workers:
api:
volumes:
- name: shared.
configMap:
name: '{{ include "workload.fullname" . }}-shared'
volumeMounts:
- name: shared.
mountPath: /etc/shared/
readOnly: trueSecret values sit in plain text in your values files and in the Helm release. To keep them out of your values files, you can:
- Inject them from CI, with
--setor a values file built from CI variables. - Encrypt your values files with helm-secrets.
- Sync them from a secret store with an operator, then reference the synced Secret through envFrom or volumes.
The library renders External Secrets resources itself from externalSecrets entries,
but since their values live in the store, a change there won't roll out your workers
unless a controller such as Reloader watches them through annotations.
Any other operator needs its resources in your own chart templates, so pick one your cluster already runs:
| Operator | API group |
|---|---|
| External Secrets | external-secrets.io |
| Secrets Store CSI Driver | secrets-store.csi.x-k8s.io |
| Vault Secrets Operator | secrets.hashicorp.com |
| Sealed Secrets | bitnami.com |
Each one registers its own API group, so you can check which are installed with:
kubectl api-versions | grep -E 'external-secrets.io|secrets-store.csi.x-k8s.io|secrets.hashicorp.com|bitnami.com'Each entry under volumeClaims requests storage and mounts it at its mount path, but how it's claimed depends on the component.
A StatefulSet worker gets one claim per pod,
while any other component shares a single PersistentVolumeClaim that survives helm uninstall when retain is set.
For example, this cron dumps the database every night into a retained claim:
crons:
backup:
schedule: "0 3 * * *"
concurrencyPolicy: Forbid
restartPolicy: OnFailure
image:
repository: postgres
tag: 17.6-alpine
command: [sh, -c, "pg_dump --host postgres.database.svc.cluster.local --format custom --file /backups/$(date +%F).dump"]
envFrom:
- secretRef:
name: database-credentials # PGDATABASE, PGUSER and PGPASSWORD
volumeClaims:
- name: backups.
mountPath: /backups/
size: 20Gi
retain: trueInit containers run in order before the component container.
Give one restartPolicy: Always
and it becomes a native sidecar (Kubernetes 1.29+),
which runs beside the component container and stops after it so that Jobs and CronJobs still complete.
Here the job waits for its database, then sends its telemetry to an OpenTelemetry Collector sidecar, which outlives the job container long enough to flush the last batch:
jobs:
migrate:
restartPolicy: OnFailure
image:
registry: registry.example.com
repository: migrate
env:
OTEL_EXPORTER_OTLP_ENDPOINT: http://localhost:4317
configMaps:
- name: otel.
mountPath: /etc/otelcol/
files: [otel/config.yaml]
initContainers:
- name: wait-db.
image: busybox:1.37.0
command: [sh, -c, "until nc -z postgres.database.svc.cluster.local 5432; do sleep 1; done"]
- name: otel-collector.
image: otel/opentelemetry-collector:0.161.0
volumeMounts:
- name: configmap-otel.
mountPath: /etc/otelcol/
restartPolicy: AlwaysTo run a job at a given step of the release rather than alongside the workers, give it a hook.
It then runs as a Helm hook,
and Helm waits for it to complete before going on.
Here the database is migrated on each install and upgrade, before the workers roll out:
jobs:
migrate:
hook:
events: [pre-install, pre-upgrade]
weight: 0
deletePolicies: [before-hook-creation, hook-succeeded]
restartPolicy: OnFailure
image:
registry: registry.example.com
repository: migrateThe job can count on its own ConfigMaps, Secrets, ExternalSecrets and ServiceAccount from its first run,
since Helm creates them just before it.
Its volume claims and the release ServiceAccount, however, come with the rest of the release,
so a pre-install job won't find them yet.
An empty image tag falls back to the appVersion value, then to the chart AppVersion.
To share one image between components, reuse it with a YAML anchor
or, since the image reference is rendered with tpl, with a template expression:
image: &image
registry: registry.example.com
repository: api
pullPolicy: &pullPolicy IfNotPresent
workers:
api:
image:
registry: "{{ .Values.image.registry }}" # through 'tpl'
repository: "{{ .Values.image.repository }}"
pullPolicy: *pullPolicy
jobs:
migrate:
image: *image # through YAML anchorPorts are declared by name, and the Service, the routes and the monitors all refer to a port through that name. Each port:
- Is TCP, unless written as
{port: 5353, protocol: UDP}. - Reaches the container as an environment variable,
http-metrics: 9090givingHTTP_METRICS_PORT=9090. - Is exposed by the Service on the container port, or on the port of the same name under
service.ports.
For example, this worker listens on 8080 and its Service serves it on 80:
workers:
api:
ports:
http: 8080
service:
ports:
http: 80An env entry is either a literal
or an EnvVarSource,
which covers what envFrom can't, like reading a pod field or renaming a single key from a Secret the chart doesn't own:
workers:
api:
env:
POD_NAME:
fieldRef:
fieldPath: metadata.name
GOMEMLIMIT:
resourceFieldRef:
resource: limits.memory
DB_PASSWORD:
secretKeyRef:
name: database-credentials
key: passwordOnce autoscaling or keda is enabled, the library drops replicaCount,
since every upgrade would otherwise reset the workload to that count and fight the autoscaler,
as Kubernetes warns.
Enabling it on a running release can briefly scale the workload down to one pod, until the autoscaler catches up.
The schema rejects values that would conflict once rendered:
autoscalingwithkeda, since KEDA creates its own HorizontalPodAutoscaler.podMonitorwithserviceMonitor, since both would scrape the same pods.replicaCount,paused,progressDeadlineSeconds,autoscalingorkedaon a DaemonSet.pausedorprogressDeadlineSecondson a StatefulSet.
A component's pods run as the serviceAccount set on that component, or as the release-wide one when it sets none.
Both take the same keys:
- With
create: true, the library creates the account asname, or under a generated name when the name is empty. - With
create: false, pods use the existingnameaccount, or the namespace default one when the name is empty. - With
rules, the library also creates a Role and binds it to the account.
The generated name is <fullname> for the release account and <fullname>-<component> for a component one,
and the matching Role and RoleBinding carry the same name.
Release-wide resources are named <fullname> as in a helm create chart, component resources <fullname>-<component>,
and resources built from a list entry <fullname>-<component>-<name>.
Each part is capped so that the longest form still fits the 63 characters of a DNS label:
- 26 characters for the fullname, which is cut.
- 20 for the component name.
- 15 for the name of a ConfigMap, Secret, ExternalSecret or volume claim entry.
- 2 for the dashes.
Pods are selected by app.kubernetes.io/name and app.kubernetes.io/instance, shared across the release,
plus app.kubernetes.io/component to tell components apart.
For your own metadata, labels and annotations go on the component controller, podLabels and podAnnotations on its pods.
Template expressions work in command, args, volumes, initContainers, envFrom, annotations, podAnnotations,
the image reference and ingressRoute.routes, which the library renders with tpl.
Besides the chart root context, they see the current component as .Component.Name and .Component.Values,
so {{ .Component.Values.env.BINARY_NAME }} reads one of its env values
and workload.componentName returns its resource name, <fullname>-<component>.
The library treats the following scalars as missing when they're set to 0, false or "", so Kubernetes applies its own default:
activeDeadlineSecondsautoscaling.minReplicasautoscaling.targetCPUUtilizationPercentageautoscaling.targetMemoryUtilizationPercentagecompletionModecompletionsconcurrencyPolicyexternalSecrets[].refreshIntervalimage.pullPolicyimage.registrykeda.maxReplicaCountminReadySecondsparallelismpausedpdb.maxUnavailablepdb.minAvailablepodManagementPolicypodMonitor.intervalpodMonitor.pathprogressDeadlineSecondsserviceMonitor.intervalserviceMonitor.pathsuspendtimeZonevolumeClaims[].storageClassNamevpa.updateMode
The following keep 0 or false, because there it means something different from leaving the key out:
automountServiceAccountTokenbackoffLimitfailedJobsHistoryLimitkeda.cooldownPeriodkeda.minReplicaCountkeda.pollingIntervalreplicaCountrevisionHistoryLimitstartingDeadlineSecondssuccessfulJobsHistoryLimitterminationGracePeriodSecondsttlSecondsAfterFinished
Every key is documented in chart/values.yaml, an example the library never reads, and chart/values.schema.json carries the same descriptions for validation and completion in your editor.
Keys typed after Kubernetes objects follow the Kubernetes API reference.