Skip to content

fix(client): replace pool members whose reconnect strategy gave up - #3525

Merged
nkaradzhov merged 1 commit into
redis:masterfrom
maxymlyskov:fix/replace-terminated-pool-members
Oct 21, 2026
Merged

nkaradzhov merged 1 commit into
redis:masterfrom
maxymlyskov:fix/replace-terminated-pool-members

Conversation

@maxymlyskov

@maxymlyskov maxymlyskov commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Description

When a pool member's reconnectStrategy gives up, the member emits 'terminated' and is closed for good, but the pool keeps handing it out. Every task on it rejects with ClientClosedError, tasks queued behind it are routed to it, pool.connect() is a no-op and pool.close() rejects, so only a new pool recovers. With socketTimeout and the default strategy this hits idle members after any quiet period longer than the timeout.

Redis 8, built dist of each side:

# master
  after 1.2 s idle: 6 pings -> ClientClosedError,ClientClosedError,ClientClosedError,ClientClosedError,ClientClosedError,ClientClosedError
  pool.close() rejected ClientClosedError
# this branch
  after 1.2 s idle: 6 pings -> PONG,PONG,PONG,PONG,PONG,PONG
  pool.close() resolved

The pool now marks a member on 'terminated' (a WeakSet next to #abandoned) and destroys it instead of reusing it. Tasks queued behind it get a replacement, and if that can't connect while nothing else is in use, they get the connect error through the rule #create already applies while closing. I used 'terminated' rather than !isOpen so a member you destroyed yourself still makes close() reject (#3437's test). execute() dispatches the task again after dropping a dead idle member, since a reassigned let loses its narrowing in the publish() callback on TS 5.3.

Not changed: a task queued at maximum behind a failed scale-up connect still waits for acquireTimeout. That happens on master without a dead member too, so I'd keep it for a separate PR.

Added 5 cases to pool.spec.ts against a real server with CLIENT KILL, they fail on master with "The client is closed". pool.spec.ts and the other specs that import pool.ts pass on Linux, tsc and eslint are clean.


Checklist

  • Does npm test pass with this change (including linting)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?

Note

Medium Risk
Changes core pool lifecycle (acquire, return, close, and scale-up) when connections are permanently terminated; behavior is covered by new integration tests but affects all pooled Redis usage after reconnect exhaustion.

Overview
Fixes Redis client pool behavior when a member’s reconnect strategy gives up: the pool no longer reuses that connection, so work stops failing with ClientClosedError and close() can complete normally.

The pool tracks permanently dead members in a #terminated WeakSet (on the client’s 'terminated' event). execute() drops terminated idle clients and retries the task; #returnClient() destroys them after use and starts a replacement when tasks are waiting; #create(true) rejects queued work if replacement connect fails with no other clients (same idea as during close). close() destroys terminated idle members instead of calling close() on them.

Five integration tests use CLIENT KILL with reconnectStrategy: false to cover idle replacement, queued tasks, failed replacement, and close paths.

Reviewed by Cursor Bugbot for commit b524c64. Bugbot is set up for automated code reviews on this repo. Configure here.

A pool member whose reconnect strategy gives up ('terminated') is closed for
good, but the pool kept handing it out: every task on it rejected with
ClientClosedError, tasks queued behind it were routed to it, and close()
rejected, until the pool was recreated.

Mark such members on 'terminated' and destroy them instead of reusing them.
Tasks queued behind one get a replacement client; if the replacement cannot
connect and no other client is in use, they are rejected with the connect
error, as #create already does while closing.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-20T15:21:56.156322Z b524c64 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@nkaradzhov nkaradzhov left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM.

@nkaradzhov
nkaradzhov merged commit 7ce0304 into redis:master Oct 21, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants