Repository navigation
Conversation
EmilienM
left a comment
There was a problem hiding this comment.
Another iteration needed: 1 blocking, 4 major, 7 minor (inline, 5 below)
The tunnel REST/GraphQL/passthrough loops are covered and tested, but the JSON-RPC/MCP loop and the forward-proxy entry path don't get the fix, and response_started can still suppress the 502 with nothing delivered to the client.
Outside the diff:
- Blocking: the PR has no area label, so Review Readiness fails;
area:supervisor(orarea:runtime) fits. - Major:
crates/openshell-supervisor-network/src/proxy.rs:6836, the forward-proxy error boundary downcasts the sibling typed relay errors but has no arm for the newUpstreamHttpError, so plain forward requests still close silently with no 502 and the allow-event for an allowed request is now skipped. - Major: issue #5322's first acceptance criterion (replaying a request sent on a reused connection when the upstream closed before any response byte) isn't implemented; since the PR closes the issue, either implement it for idempotent methods or update the issue so the descope is recorded.
- Minor:
relay.rs:2355, an upstream failure now ends the websocket middleware session asCancellationwhile the sibling arms useUpstreamFailure. - Minor: the PR body says the connection behavior is documented, but the second commit removed the docs paragraph; restore a trimmed note or drop the claim.
How this was reviewed
- Commit
e5fe0357, round 1 - Standard review, because the change is code in
crates/openshell-supervisor-network/src/l7/relay.rs,l7/rest.rs, andl7/rest/http_response.rs - Passes: architecture, performance, security, scalability, rust, supply-chain, then correctness
- Independent skeptics checked each candidate finding (2 per blocking or major, 1 per minor); 11 of 11 held up, 2 of them after re-checking the code
- Repo rules read from
main: AGENTS.md, CLAUDE.md, CONTRIBUTING.md, docs/CONTRIBUTING.mdx - Not covered: nothing skipped
|
All five HTTP loops now watch idle upstream connections, including JSON-RPC/MCP. Forward HTTP uses the same upstream-error handling and preserves allowed-request telemetry. Response commitment tracks successful client writes, so an incomplete buffered header can still produce a 502. Original I/O causes are preserved, and truncated bodies are distinguished from policy or middleware failures after normal body completion. The shared handler preserves the request ID, shuts down upstream, and records a 502 only after delivery succeeds. Session cleanup uses UpstreamFailure. The tests now cover inspected REST, resets and the forward-handler path. The stale docs claim is removed, and |
BlockedGitHub reports merge conflicts with Action required: @shiju-nv, resolve the merge conflicts and push the updated branch. Gator will re-check the process gates and review history on the next cycle. Gator metadata
|
FrostGod
left a comment
There was a problem hiding this comment.
Some clients, such as Go's net/http, automatically retry an idempotent request (like GET) when a reused connection fails before any response arrives. A 502 counts as a real response, so those clients will now show the error instead of retrying. Is that trade-off intended for GET traffic, or should the 502 apply only to non-idempotent requests?
| // Parsing wins a simultaneous close so forwarding can return 502, but | ||
| // ready unsolicited data must never become that request's response. | ||
| if matches!(request, Ok(Some(_))) { | ||
| match upstream.read(&mut byte).now_or_never() { |
There was a problem hiding this comment.
When a request is ready and the upstream has sent unsolicited bytes, the client gets a bare close. The request was never forwarded, so could we return a 502 here instead?
| } | ||
| return Ok(RelayOutcome::Consumed); | ||
| return Err(miette::Report::new(UpstreamHttpError { | ||
| response_started: informational_header_bytes > 0, |
There was a problem hiding this comment.
Any 1xx response, including the 100 Continue the proxy writes itself for SigV4 (rest.rs:1077), marks the response as started, so a later upstream failure gets no 502. HTTP still allows a final status after a 1xx. Should "started" count only bytes of the final response?
| U: AsyncRead + Unpin, | ||
| { | ||
| let mut byte = [0u8; 1]; | ||
| let request = tokio::select! { |
There was a problem hiding this comment.
If the upstream closes while the client is still sending request headers, the partial request is dropped with no reply. Is that intended? If so, could we add a short comment saying so?
| request | ||
| } | ||
|
|
||
| fn emit_idle_upstream_close(ctx: &L7EvalContext, result: std::io::Result<usize>) { |
There was a problem hiding this comment.
Every normal keep-alive timeout now emits an Informational OCSF "Close" event. With upstreams that time out after 5 seconds, that could be a lot of events. Is that volume OK, or should this be a debug! log?
Signed-off-by: Shiju <shiju@nvidia.com>
c548047 to
5e7eaca
Compare
Compatibility review (advisory)New or updated breaking changes; see the full report for all findings. 1. api-behavior — No explicit risk acceptance foundRelay-owned HTTP failures while awaiting a final response now return
Review details
Advisory review; publication gates are unchanged. |
johntmyers
left a comment
There was a problem hiding this comment.
gator-agent
PR Review Status
@shiju-nv, I checked your update about the five HTTP loops, forward-proxy handling, response commitment, and failure telemetry against the current patch. The implementation and Fern docs follow the updated issue scope, including explicit 502 responses for GET and no automatic replay. One blocking test update remains: Branch Checks fails three existing response/tracing tests on Linux and macOS because they still expect the previous behavior.
Action required: @shiju-nv, update those regression tests as described inline and push the change so CI can verify the new contract.
Blocking findings:
GATOR-5e7eacaa-01: Existing EOF and request-tracing assertions must match the new error and event handling.
Carried findings: None.
Non-blocking suggestions: None.
Gator metadata
- Validation: Focused networking bug fix implementing issue #5322.
- Docs: Fern network-rules page explains idle closure, 502 responses, and retry implications.
- Checks: DCO and issue acceptance pass; required Branch Checks fails three existing Rust tests on Linux and macOS.
- E2E: Required for network-proxy behavior; test:e2e is absent and E2E execution was skipped. Dispatch follows resolution of blocking review feedback.
- Head SHA:
5e7eacaaedede863ec39ef619711eb03d5a56293 - Base SHA:
ce58cf8e9b953281352d98a637e41616295b1318 - Merge base SHA:
0be63c5b0de27692df03ba0e4b3bc212312d3b3c - Patch ID:
d5ff900a5c311c8b47cf09e6c8c53f7bb967c679 - Gator payload:
13 - Review mode:
initial - Previous reviewed SHA:
none - Review budget exhausted:
no - Maintainer decision required:
no - Next state:
gator:in-review
Signed-off-by: Shiju <shiju@nvidia.com>
|
/ok to test edce16d |
|
Label |
PR Review Status@shiju-nv, I checked your updated EOF and tracing tests against the new error handling. The direct helper now asserts the typed upstream failure, forward HTTP checks the correlated 502 and transport-failure event, and the startup test exercises malformed framing at the outer error boundary. Required CI is now queued or running on this head, including Branch Checks, Helm Lint, and Branch E2E Checks. Gator will inspect the results next cycle. Blocking findings: None. Carried findings: None; Gator metadata
|
Summary
Retire HTTP connections promptly when the upstream closes while idle, including JSON-RPC and MCP. For relay-owned upstream failures, return a correlated
502 Bad GatewaywithConnection: closebefore any final response bytes are written to the client; close without appending another status once a final response has started. Apply the same error handling to plain forward HTTP and CONNECT tunnels.Related Issue
Closes #5322
Scope follows the updated issue criteria and the issue discussion. Automatic reconnect/replay remains outside this fix. Explicit gateway errors apply to every HTTP method, including GET; applications may need to retry a 502 because some clients only retry transport-level failures.
Changes
Release Notes
HTTP clients receive idle upstream closure promptly. Relay-owned failures before a final response starts return an explicit gateway error with a request ID.
Testing
cargo clippy --locked -p openshell-supervisor-network --lib --tests -- -D warnings, Rust formatting, whitespace and SPDX checks passed.Validated with Rust 1.95.0 on macOS after rebasing onto
0be63c5b0and resolving conflicts with current request tracing and streaming middleware.Checklist