Repository navigation
Fix health check worker never acknowledging ProcSignalBarrier (DROP DATABASE hang) - #1196
Merged
Merged
Conversation
…lcheck hang) LatchWait() -- the sole sleep point in both the health-check launcher's and each per-database health-check worker's main loop -- called WaitLatch() and ResetLatch() but never CHECK_FOR_INTERRUPTS(). A background worker only acknowledges a ProcSignalBarrier (the mechanism DROP DATABASE uses to wait for every backend to release smgr state) from inside CHECK_FOR_INTERRUPTS(), so as long as any health-check worker was running, every DROP DATABASE anywhere in the cluster -- including pg_regress's own schedule setup -- blocked forever, logging "still waiting for backend ... to accept ProcSignalBarrier" every 5s. Found while working on an unrelated change, but reproduces on an unmodified checkout against Postgres 17.11, so it's a pre-existing issue: as long as any pgautofailover health-check worker is running anywhere in the cluster (which is always, by default), every DROP DATABASE blocks forever. Confirmed fixed: the full monitor regression schedule (20 tests) and isolation suite (6 tests) now complete and pass against a live PG17 instance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LatchWait()-- the sole sleep point in both the health-check launcher's and every per-database health-check worker's main loop -- callsWaitLatch()+ResetLatch()but neverCHECK_FOR_INTERRUPTS().A background worker only acknowledges a
ProcSignalBarrier(the mechanismDROP DATABASEuses to wait until every backend in the cluster has released smgr state) from insideCHECK_FOR_INTERRUPTS()--WaitLatch()wakes the process up on the pending signal, but doesn't process it itself.Practical effect: as long as any pgautofailover health-check worker is running anywhere in the cluster (which is always, by default, once the extension is loaded via
shared_preload_libraries), everyDROP DATABASEstatement anywhere in that cluster blocks forever, repeatedly logging:This includes
pg_regress's own schedule setup, somake installcheckagainst a live Postgres 17 instance hangs indefinitely before running a single test.Root cause
health_check_worker.c'sLatchWait()is the common per-iteration sleep point for both worker loops. Standard Postgres background worker loops callCHECK_FOR_INTERRUPTS()right after waking fromWaitLatch()/ResetLatch()precisely so that pending interrupts -- including aProcSignalBarrier-- get processed promptly. This worker never did.Fix
Add
CHECK_FOR_INTERRUPTS()right afterResetLatch()inLatchWait(), matching the idiom used by every other Postgres background worker.Reproduction / verification
Reproduces on an unmodified checkout against Postgres 17.11:
CREATE DATABASEand ordinary queries work fine, but anyDROP DATABASEagainst a cluster withshared_preload_libraries = 'pgautofailover'loaded hangs forever once the extension's health-check worker has started.With the fix: rebuilt and reinstalled the extension against a live PG17.11 instance; the full monitor regression schedule (20 tests) and isolation test suite (6 tests) now complete and pass via
make installcheck, where they previously hung.Test plan
make installcheckcompletes (was hanging indefinitely before this fix)