Skip to content

feat(run-engine): trigger tasks pinned to an external deployment id - #4664

Open
0ski wants to merge 1 commit into
oskar/feat-external-id-deployfrom
oskar/feat-external-id-trigger
Open

feat(run-engine): trigger tasks pinned to an external deployment id#4664
0ski wants to merge 1 commit into
oskar/feat-external-id-deployfrom
oskar/feat-external-id-trigger

Conversation

@0ski

@0ski 0ski commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

The SDK discovers an external deployment id at runtime (explicit TRIGGER_EXTERNAL_DEPLOYMENT_ID always; platform commit-SHA variables and generic fallbacks when TRIGGER_AUTOMATIC_SKEW_VERSION_PROTECTION=1) and sends it alongside lockToVersion; the server resolves precedence (version > external id > current). An id held by a deployed deployment pins the run to that worker; an in-flight or unknown id parks the run in PENDING_VERSION with the id in TaskRun.annotations, wakes it pinned when a deployment carrying the id finalizes (ClickHouse candidates, Postgres authoritative), and expires it after a deadline that re-checks Postgres before acting. Parking outranks delaying and preserves delayUntil. The id is projected to ClickHouse task_runs_v2.external_deployment_id during replication. Redis cache for id-to-worker resolution, guarded version-aware writes.

Ids are not unique. Several deployments can hold one id - a --force rebuild is the ordinary way to get there - so resolution always picks the highest version among the candidates, never the newest by timestamp. The rule is applied identically on both paths that can bind a run to a worker: resolveExternalDeployment at trigger time, and PendingVersionSystem when a landing deployment wakes a parked run. Version comparison is numeric on the counter half, so 20260807.10 outranks 20260807.9.

A run whose id never lands expires at the deadline with EXTERNAL_DEPLOYMENT_NOT_FOUND and an error naming the id it waited for, which is what a failed build or a typo looks like from the caller. Default deadline is one hour (EXTERNAL_DEPLOYMENT_PARK_DEADLINE_MS).

Debounce registration happens in both the parked and the delayed branch through one helper, so a debounced run that parks still binds its debounce key; without it every later trigger for the same key created another parked run, and all of them executed when the deployment landed. The two DELAYED-only status checks in DebounceSystem also accept PENDING_VERSION, without which the lock-contention fallback would rethrow a 5xx the SDK retries and amplifies, and the fast path would push every trigger on a parked key through the redlock.

Resolution is skipped in development. A dev environment cannot hold a WorkerDeployment - trigger dev registers a BackgroundWorker with nothing behind it, and deploy --env refuses dev - so an external deployment id there could only ever park, and the parked run then expired against the dev TTL while a connected dev worker sat idle. The id is still annotated so the dashboard shows what the app sent (TRI-13000).

@changeset-bot

changeset-bot Bot commented Aug 18, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: b919380

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 27 packages
Name Type
@trigger.dev/core Patch
@trigger.dev/sdk Patch
@trigger.dev/build Patch
trigger.dev Patch
@trigger.dev/python Patch
@trigger.dev/redis-worker Patch
@trigger.dev/schema-to-json Patch
@internal/clickhouse Patch
@internal/llm-model-catalog Patch
@internal/metrics-pipeline Patch
@trigger.dev/rbac Patch
@internal/redis Patch
@internal/replication Patch
@internal/run-engine Patch
@internal/run-store Patch
@internal/schedule-engine Patch
@internal/tracing Patch
@internal/webhook-engine Patch
@internal/webhook-sources Patch
@internal/dashboard-agent Patch
@internal/cache Patch
@trigger.dev/react-hooks Patch
@trigger.dev/rsc Patch
@trigger.dev/database Patch
@trigger.dev/otlp-importer Patch
@trigger.dev/sso Patch
@internal/testcontainers Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 45aa0519-590a-41cc-ba12-9f0463b21e16

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@0ski
0ski marked this pull request as ready for review August 18, 2026 13:15
devin-ai-integration[bot]

This comment was marked as resolved.

Comment thread .changeset/trigger-external-deployment-id.md Outdated
@0ski
0ski force-pushed the oskar/feat-external-id-trigger branch from 8f7617a to bb5d381 Compare August 18, 2026 18:50
@pkg-pr-new

pkg-pr-new Bot commented Aug 18, 2026

Copy link
Copy Markdown

Open in StackBlitz

@trigger.dev/build

npm i https://pkg.pr.new/@trigger.dev/build@b919380

trigger.dev

npm i https://pkg.pr.new/trigger.dev@b919380

@trigger.dev/core

npm i https://pkg.pr.new/@trigger.dev/core@b919380

@trigger.dev/python

npm i https://pkg.pr.new/@trigger.dev/python@b919380

@trigger.dev/react-hooks

npm i https://pkg.pr.new/@trigger.dev/react-hooks@b919380

@trigger.dev/redis-worker

npm i https://pkg.pr.new/@trigger.dev/redis-worker@b919380

@trigger.dev/rsc

npm i https://pkg.pr.new/@trigger.dev/rsc@b919380

@trigger.dev/schema-to-json

npm i https://pkg.pr.new/@trigger.dev/schema-to-json@b919380

@trigger.dev/sdk

npm i https://pkg.pr.new/@trigger.dev/sdk@b919380

commit: b919380

devin-ai-integration[bot]

This comment was marked as resolved.

@0ski
0ski force-pushed the oskar/feat-external-id-trigger branch from bb5d381 to 2fac2c0 Compare August 19, 2026 10:37
devin-ai-integration[bot]

This comment was marked as resolved.

@0ski
0ski force-pushed the oskar/feat-external-id-trigger branch from 2fac2c0 to 697f125 Compare August 19, 2026 11:29
devin-ai-integration[bot]

This comment was marked as resolved.

@0ski
0ski force-pushed the oskar/feat-external-id-trigger branch from 697f125 to cc53ad2 Compare August 19, 2026 11:58
devin-ai-integration[bot]

This comment was marked as resolved.

The SDK discovers an external deployment id at runtime (explicit TRIGGER_EXTERNAL_DEPLOYMENT_ID always; platform commit-SHA variables and generic fallbacks when TRIGGER_AUTOMATIC_SKEW_VERSION_PROTECTION=1) and sends it alongside lockToVersion; the server resolves precedence (version > external id > current). An id held by a deployed deployment pins the run to that worker; an in-flight or unknown id parks the run in PENDING_VERSION with the id in TaskRun.annotations, wakes it pinned when a deployment carrying the id finalizes (ClickHouse candidates, Postgres authoritative), and expires it after a deadline that re-checks Postgres before acting. Parking outranks delaying and preserves delayUntil. The id is projected to ClickHouse task_runs_v2.external_deployment_id during replication. Redis cache for id-to-worker resolution, guarded version-aware writes.

Ids are not unique. Several deployments can hold one id - a --force rebuild is the ordinary way to get there - so resolution always picks the highest version among the candidates, never the newest by timestamp. The rule is applied identically on both paths that can bind a run to a worker: resolveExternalDeployment at trigger time, and PendingVersionSystem when a landing deployment wakes a parked run. Version comparison is numeric on the counter half, so 20260807.10 outranks 20260807.9.

A run whose id never lands expires at the deadline with EXTERNAL_DEPLOYMENT_NOT_FOUND and an error naming the id it waited for, which is what a failed build or a typo looks like from the caller. Default deadline is one hour (EXTERNAL_DEPLOYMENT_PARK_DEADLINE_MS).

Debounce registration happens in both the parked and the delayed branch through one helper, so a debounced run that parks still binds its debounce key; without it every later trigger for the same key created another parked run, and all of them executed when the deployment landed. The two DELAYED-only status checks in DebounceSystem also accept PENDING_VERSION, without which the lock-contention fallback would rethrow a 5xx the SDK retries and amplifies, and the fast path would push every trigger on a parked key through the redlock.

Resolution is skipped in development. A dev environment cannot hold a WorkerDeployment - trigger dev registers a BackgroundWorker with nothing behind it, and deploy --env refuses dev - so an external deployment id there could only ever park, and the parked run then expired against the dev TTL while a connected dev worker sat idle. The id is still annotated so the dashboard shows what the app sent (TRI-13000).
@0ski
0ski force-pushed the oskar/feat-external-id-trigger branch from cc53ad2 to b919380 Compare August 19, 2026 12:44

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review

Comment thread internal-packages/run-engine/src/engine/systems/debounceSystem.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants