Support passive monitoring of Postgres locks - #285
Conversation
30bc1b1 to
d689cf1
Compare
d689cf1 to
295523c
Compare
| When `HandleLostToken` is used on a lock backed by a library-owned connection, the library monitors the connection passively using `NpgsqlConnection.WaitAsync`, which uses a blocking socket read that detects connection loss (e.g. a database restart or `pg_terminate_backend`) as soon as the socket breaks, without executing any query. The monitored session therefore shows as `idle` in `pg_stat_activity`. | ||
|
|
||
| Two things to be aware of: | ||
| - Because the monitored session is idle, server-side idle-session reapers (`idle_session_timeout`, `idle_in_transaction_session_timeout`, or aggressive gateways) can kill it. If any of these are in play, set `KeepaliveCadence`, as when monitoring is active, the keepalive query will be interleaved with the passive wait. |
There was a problem hiding this comment.
Does this issue exist if pg_sleep is used? Trying to understand if this is something that could become an issue for folks using the library today if they upgrade and get this behavior.
There was a problem hiding this comment.
No it doesn't impact pg_sleep as that keeps the connection active. With the updated logic where we always do a keepalive per minute, it would only impact consumers that have an idle_session_timeout of <60 which is very unlikely.
|
Thanks for filing @Hawxy ! I believe this change makes sense overall but I'd like to change the approach a bit to be more thread-safe and leverage the existing abstractions a bit more. Please take a look at https://github.com/madelson/DistributedLock/pull/285/changes#r3839331256 and let me know if you have questions or thoughts. |
|
Changes have been made per comments. |
2 participants
Hiya, I've used this package quite a lot over the last few years, both in my work as well as open source (I moved MartenDB over to it a while ago, and it'll be used within the v1 of Wallaby). I find the lock monitoring especially useful and it eliminated an entire class of problems for us, but the
pg_sleepquery causes a few issues:pg_sleepnot doing anything, to the point that we provided an opt-out on monitoring just to eliminate the noise. This gets flagged by automated problem finders too, like Datadog's DBM and RDS Performance InsightsThis PR adds a passive approach that waits for the socket to break instead. In monitoring tools this will simply show up as an idle connection. This is codepath is enabled by default, but is unsupported when:
If either case is true, it'll fall back to using the pg_sleep path.