Self-hosting: using Kubernetes Secrets instead of hardcoded values is not working #2859

Activity

1. matt-aitken

   changed the title [-]bug: Using Kubernetes Secrets instead of hardcoded values is not working[/-] [+]Self-hosting: using Kubernetes Secrets instead of hardcoded values is not working[/+] on Jan 9, 2026
2. matt-aitken commented on Jan 9, 2026

   

   matt-aitken

   on Jan 9, 2026

   Member

   More actions

   @TheRealFloatDev can you provide some version numbers and more details about your setup and where you're deploying to?

   We have a lot of self-hosters using this setup.
3. TheRealFloatDev commented on Jan 9, 2026

   

   TheRealFloatDev

   on Jan 9, 2026

   Last edited by TheRealFloatDev

   Author

   More actions

   Sure.

   Regarding "where" I am deploying to:
   I am deploying to a RKE2 Kubernetes Cluster, running Hetzner Cloud AMD64 servers.

   Regarding the version
   I am using the docs-provided install command which automatically uses the latest version of 4.0.

   helm upgrade -n trigger --install trigger \
     oci://ghcr.io/triggerdotdev/charts/trigger \
     --version "~4.0.0" \
     -f trigger-production-values.yaml

   The version it installed is v4.0.5.

   Exact content of the trigger-production-values.yaml

   # REQUIRED: Generate your own secrets using: openssl rand -hex 16
   secrets:
     enabled: false # False to use existing secret - true to specify them here directly (only for testing/dev)
     existingSecret: "trigger-secret" # Name of the Kubernetes secret created from trigger-secret.yaml
   
   # Production webapp configuration
   webapp:
     # Origin configuration
     appOrigin: "REDACTED" # Not redacted in the actual file
     loginOrigin: "REDACTED" # Not redacted in the actual file
     apiOrigin: "REDACTED" # Not redacted in the actual file
   
     # Production ingress
     ingress:
       enabled: true
       className: "nginx"
       annotations:
         cert-manager.io/cluster-issuer: "letsencrypt-prod"
         nginx.ingress.kubernetes.io/ssl-redirect: "true"
       hosts:
         - host: "REDACTED" # Not redacted in the actual file
           paths:
             - path: /
               pathType: Prefix
       tls:
         - secretName: trigger-tls
           hosts:
             - "REDACTED" # Not redacted in the actual file
   
     resources:
       limits:
         cpu: 2000m
         memory: 4Gi
       requests:
         cpu: 500m
         memory: 1Gi
   
   # Production PostgreSQL (or use external)
   postgres:
     primary:
       persistence:
         enabled: true
         size: 20Gi # Keep an eye on this depending on your usage
         storageClass: "longhorn"
       resources:
         limits:
           cpu: 1000m
           memory: 2Gi
         requests:
           cpu: 500m
           memory: 1Gi
   
   # Production Redis (or use external)
   redis:
     master:
       persistence:
         enabled: true
         size: 10Gi # Keep an eye on this depending on your usage
         storageClass: "longhorn"
       resources:
         limits:
           cpu: 500m
           memory: 1Gi
         requests:
           cpu: 250m
           memory: 512Mi
   
   # Production ClickHouse
   clickhouse:
     # Set to true to enable TLS/secure connections in production
     secure: false
     persistence:
       enabled: true
       size: 10Gi # Keep an eye on this depending on your usage
       storageClass: "longhorn"
     # ClickHouse can be very resource intensive, so we recommend setting limits and requests accordingly
     # Note: not doing this can cause OOM crashes which will cause issues across many different
     resources:
       limits:
         cpu: 4000m
         memory: 16Gi
       requests:
         cpu: 2000m
         memory: 8Gi
   
   # Production S3-compatible object storage
   minio:
     deploy: false # Disable MinIO deployment in production
   s3:
     deploy: false # Disable built-in S3 deployment in production (should be the minio setting above, but just making sure)
     external:
       endpoint: "REDACTED" # Not redacted in the actual file
       existingSecret: "trigger-secret" # Use the same secret created from trigger-secret.yaml (contains OBJECT_STORE_ACCESS_KEY_ID and OBJECT_STORE_SECRET_ACCESS_KEY)
       existingSecretAccessKeyIdKey: "OBJECT_STORE_ACCESS_KEY_ID"
       existingSecretSecretAccessKeyKey: "OBJECT_STORE_SECRET_ACCESS_KEY"
   
   # Production Registry
   registry:
     deploy: true # Use the built-in Docker registry for storing images
     auth:
       username: "registry-user"
       password: "your-strong-registry-password"
     persistence:
       enabled: true
       size: 45Gi # Keep an eye on this depending on your usage
       storageClass: "longhorn"
   
     # Production ingress
     ingress:
       enabled: true
       className: "nginx"
       annotations:
         cert-manager.io/cluster-issuer: "letsencrypt-prod"
         nginx.ingress.kubernetes.io/ssl-redirect: "true"
       hosts:
         - host: "REDACTED" # Not redacted in the actual file
           paths:
             - path: /
               pathType: Prefix
       tls:
         - secretName: registry-tls
           hosts:
             - "REDACTED" # Not redacted in the actual file
   
   # Production Supervisor (Kubernetes worker orchestrator)
   supervisor:
     resources:
       limits:
         cpu: 500m
         memory: 1Gi
       requests:
         cpu: 250m
         memory: 512Mi
   
   # Telemetry configuration
   telemetry:
     enabled: false # Disable telemetry in production

   I created this file based on the following resources:

   • https://trigger.dev/docs/self-hosting/kubernetes
   • https://github.com/triggerdotdev/trigger.dev/blob/main/hosting/k8s/helm/values-production-example.yaml
   • https://github.com/triggerdotdev/trigger.dev/blob/main/hosting/k8s/helm/values.yaml

   Secret is created like this

   apiVersion: v1
   kind: Secret
   metadata:
     name: trigger-secret
     namespace: trigger
   stringData:
     # Generate secrets using `openssl rand -hex 16`
     SESSION_SECRET:
     MAGIC_LINK_SECRET:
     ENCRYPTION_KEY:
     MANAGED_WORKER_SECRET:
     OBJECT_STORE_ACCESS_KEY_ID:
     OBJECT_STORE_SECRET_ACCESS_KEY:

   Of course with the stringData actually populated
4. nicktrn commented on Jan 9, 2026

   

   nicktrn

   on Jan 9, 2026

   Collaborator

   More actions

   @claude I think this should be working, please check why it might not be

   👀React with 👀1Reacted by claude
5. claude commented on Jan 9, 2026

   

   claudeBot

   on Jan 9, 2026 – with Claude

   Last edited by claude

   Contributor

   More actions

   Claude finished @nicktrn's task in 2m 9s —— View job • claude/issue-2859-20260109-1126 • Create PR ➔

   ---

   Fixed: Support secrets.existingSecret for Core Secrets

   • Examine Helm chart templates to understand how secrets.existingSecret is used
   • Check if environment variables are properly populated from the existing secret
   • Identify the root cause of why secrets aren't being recognized
   • Implement the fix

   Root Cause

   The bug was in webapp.yaml and supervisor.yaml. The templates only checked {{- if .Values.secrets.enabled }} before populating environment variables from secrets. When users set secrets.enabled: false to use existingSecret, the critical environment variables (SESSION_SECRET, MAGIC_LINK_SECRET, ENCRYPTION_KEY, MANAGED_WORKER_SECRET) were never set.

   Fix Applied

   Changed the condition from:

   {{- if .Values.secrets.enabled }}

   To:

   {{- if or .Values.secrets.enabled .Values.secrets.existingSecret }}

   This allows the environment variables to be populated from either:

   1. The chart-generated secret (when secrets.enabled: true)
   2. A user-provided existing secret (when secrets.existingSecret is set)

   The trigger-v4.secretsName helper already correctly returns the existingSecret name when provided.

   Files Changed

   • hosting/k8s/helm/templates/webapp.yaml - Fixed condition for SESSION_SECRET, MAGIC_LINK_SECRET, ENCRYPTION_KEY, MANAGED_WORKER_SECRET
   • hosting/k8s/helm/templates/supervisor.yaml - Fixed condition for MANAGED_WORKER_SECRET

   ---

   |
6. claude

   added a commit that references this issue on Jan 9, 2026

   fix(helm): support secrets.existingSecret for core secrets

   ...

   This issue will close once commit 61a0d61 is merged into the 'main' branch.

   61a0d61
7. nicktrn commented on Jan 9, 2026

   

   nicktrn

   on Jan 9, 2026

   Last edited by nicktrn

   Collaborator

   More actions

   hey @TheRealFloatDev there's a helm prerelease here, any chance you could check this fixes it? will merge when you give the go ahead!

   #2860 (comment)

   🧭 Helm Chart Prerelease Published

   Version: 4.0.5-pr2860.012b943

   Install:

   helm upgrade --install trigger \
     oci://ghcr.io/triggerdotdev/charts/trigger \
     --version "4.0.5-pr2860.012b943"

   ⚠️ This is a prerelease for testing. Do not use in production.
8. nicktrn

   added

   waiting-for-reply

   on Jan 10, 2026
9. nicktrn commented on Jan 14, 2026

   

   nicktrn

   on Jan 14, 2026

   Collaborator

   More actions

   will go ahead and get this merged
10. nicktrn

    closed this as completedin 260fb7con Jan 14, 2026
11. pull

    added a commit that references this issue on Jan 14, 2026

    fix(helm): support secrets.existingSecret for core secrets (triggerdo…

    ...

    This issue will close once commit 260fb7c is merged into the 'main' branch.

    260fb7c