fix: handle stop during startup with safe state transitions and hard-exit - #639
fix: handle stop during startup with safe state transitions and hard-exit#639jumski wants to merge 2 commits into
Conversation
|
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
|
View your CI Pipeline Execution ↗ for commit 943b2dd
💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗ ☁️ Nx Cloud last updated this comment at |
- Serialize Worker.stop() behind startup and allow Created -> Stopping cancellation - Add idempotent Supabase stop gate: awaits in-flight worker replacement, marks after startup settles, keeps the shutdown signal aborted, and never lets a sql close failure mask the operation error - Exit immediately with code 0 on the first process signal during startup and honor optional process.off - Document startup-signal behavior for Node/Bun process workers Fixes #635 Fixes #636 Fixes #634
…failures Review follow-up for PR 633: - Supabase shutdown runs under a 5s deadline, drains the worker before marking, force-closes SQL at the deadline, and reports dual peer failures via AggregateError without masking SQL close failures - HTTP startup waits for in-flight replacements; stopPromise permanently closes startup admission - Worker main loop retries heartbeat/batch failures with capped exponential backoff (100ms-5s, abort-aware, reset on success) - StepTaskPoller rethrows poll failures so Worker can distinguish them from empty polls - Process adapter preserves startup/drain/marking errors over cleanup failures Adds a draft news article for the 0.15.0 release (hidden on pgflow.dev until draft: false).
530448c to
943b2dd
Compare
621f55e to
75cf582
Compare
🔍 Preview Deployment: Website✅ Deployment successful! 🔗 Preview URL: https://pr-639.pgflow.pages.dev 📝 Details:
_Last updated: _ |

This PR fixes a race condition where calling
stop()on a worker during its startup phase could trigger an invalid state transition, since the lifecycle was being moved toStoppingbefore startup had a chance to complete its own transitions.Changes
Core worker shutdown sequencing
Worker.stop()now aborts the worker immediately but defers thetransitionToStopping()call until any in-flightstartupPromisehas settled. This prevents an illegalStarting → Stoppingtransition while still ensuring the abort signal prevents any batch processing from starting.The
Created → Stoppingtransition is now explicitly allowed inWorkerState, enabling workers that are stopped before they ever start to reachStoppedcleanly.Lifecycle hardening
The
workerRowguard inacknowledgeStop()has been removed from bothWorkerLifecycleandFlowWorkerLifecycle. A worker stopped before its first HTTP request never has aworkerRow, and the guard was incorrectly blocking a valid shutdown path.SupabasePlatformAdapterstop sequencingstopWorker()is now idempotent — concurrent callers share a singleperformStopWorker()promise so SQL closes exactly once. The stop sequence now:worker.stop()markWorkerStoppedhas been moved out of the shutdown handler and intoperformStopWorker()so it runs in the correct order relative toworker.stop()andsql.end(). SQL close errors are logged but do not suppress an earlier operation error.The
abortControlleris no longer reset during worker replacement once a stop has been initiated, preventing a replacement from making the platform signal live again mid-shutdown.ProcessPlatformAdapterearly exit during hung startupA
startupCompletedflag is tracked. If a signal arrives before startup finishes, the adapter now hard-exits immediately with code0rather than waiting on a potentially hung bootstrap. The docs are updated to reflect this behavior.processDepsoptionaloffSignalprocess.offis no longer required forgetProcessDeps()to succeed.offSignalis exposed asundefinedwhenprocess.offis absent, allowing environments that only implementprocess.onto work correctly.Fixes #635
Fixes #636
Fixes #634