Skip to content

Say where a Search Native exchange failed before its response - #167

Merged
liran merged 2 commits into
mainfrom
fix/name-native-exchange-failures
Nov 10, 2026
Merged

liran merged 2 commits into
mainfrom
fix/name-native-exchange-failures

Conversation

@liran

@liran liran commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

In production, a few Search Natives at a time end with UNAVAILABLE "Native HTTP exchange incomplete" while OpenSearch reports no errors and the pooled record path never fails. Every Native opens its own connection, so connection setup (2s connect and TLS handshake budgets) is the main suspect, but the failure drops the underlying error and cannot tell a handshake timeout from a reset or a server that closed the connection.

The failure now names the phase the exchange reached (connecting, TLS handshake, sending, waiting for the response head) and, when known, whether the connection was refused, reset, closed or timed out. The words come from a fixed vocabulary and never quote the error, which carries the backend URL. Behaviour, completions and failure codes are unchanged.

@liran liran self-assigned this Nov 10, 2026
net/http calls WroteRequest even when the write fails, so a connection
dropped mid-upload read as waiting for the response head. Advance only on a
successful write, drop the connect and handshake-done hooks that a losing
parallel dial can report out of order, name a failure while reading the head,
and treat a broken pipe as closed.
@liran
liran merged commit 31656bc into main Nov 10, 2026
8 checks passed
@liran
liran deleted the fix/name-native-exchange-failures branch November 10, 2026 05:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant