fix(openai): stream the Responses API for the Codex OAuth backend - #7
fix(openai): stream the Responses API for the Codex OAuth backend#7Guykaganovsky1 wants to merge 2 commits into
Conversation
The ChatGPT Codex backend (`chatgpt.com/backend-api/codex`) refuses a unary
Responses call with HTTP 400 `{"detail":"Stream must be set to true"}` — it
only ever speaks SSE. Both `invoke` and `stream` route through
`invoke_responses`, which sent `stream: None`, so every codex-OAuth turn failed
before reaching the model.
Two defects, one behind the other:
1. The request never asked for a stream. `responses_requires_stream` now latches
on the first such 400, retries that call with `stream: true`, and sends it up
front thereafter — so exactly one call pays the probe and no other
`/responses` backend changes behaviour.
2. With streaming on, the reply came back empty. Codex ends the stream with a
`response.completed` whose `output` array is EMPTY and delivers the content
in earlier `response.output_item.done` events. The fold now collects those
items and grafts them onto the terminal response. Content-type is treated as
a hint, not a contract: codex omits `text/event-stream`, so the body is
parsed as JSON first and folded as SSE whenever that fails.
Verified end to end against the live backend: `gpt-5.5` and `gpt-5.6-sol` both
return their reply through a real chat turn; before the change both failed with
the 400 above.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: Warning Your free Security trial is over. An organization admin can upgrade to Advanced for continuous pull request security review or dismiss this notice. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
How this change flows1 changed behaviour across 5 relationships. 5 surrounding behaviours are shown (60 graph nodes walked). 46 further behaviours left out to keep the diagram readable. flowchart LR
n0["OpenAiModel<br/>changed"]:::changed
n1["translate_request"]:::impacted
n2["model"]:::impacted
n3["Result"]:::impacted
n4["ModelRequest"]:::impacted
n5["..._continuation_format_and_provider_options"]:::impacted
n1 -->|uses| n3
n1 -->|uses| n4
n2 -->|uses| n0
n5 -->|calls| n2
n5 -->|tests| n2
classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff
Green: changed behaviour. Grey: surrounding behaviour. Arrows name the call, use, implementation, or test relationship. Orange: has findings. Red: has a finding that blocks the merge. |
![tinysweeper[bot]](https://avatars.githubusercontent.com/in/4526585?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dc2495ef62
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| _ => { | ||
| if let Some(response) = event.get("response") { | ||
| fallback = Some(response.clone()); |
There was a problem hiding this comment.
When an HTTP-200 Responses stream terminates with response.failed, this default arm saves its response as a successful fallback; a standalone error event is ignored and leaves an earlier partial response as the fallback. invoke_responses consequently returns a possibly empty ModelResponse, and stream() emits Completed, making provider, quota, or safety failures indistinguishable from successful output. Detect these event types and propagate a normalized provider failure instead.
AGENTS.md reference: AGENTS.md:L43-L45
Useful? React with 👍 / 👎.
| items.push(item.clone()); | ||
| } | ||
| } | ||
| "response.completed" | "response.incomplete" => { |
There was a problem hiding this comment.
When generation reaches a limit or content filter and the backend emits response.incomplete, this branch treats it identically to response.completed; parse_responses_response then hardcodes the normalized finish reason to stop. Consumers can therefore accept or cache truncated text or malformed structured output as a clean completion. Preserve the terminal status and incomplete_details.reason in the normalized finish reason instead.
AGENTS.md reference: AGENTS.md:L43-L45
Useful? React with 👍 / 👎.
| } | ||
| } | ||
| } | ||
| let mut response = final_response.or(fallback)?; |
There was a problem hiding this comment.
If the connection closes cleanly after response.created or another partial event, response.text() succeeds and this fallback returns the last in_progress response. The Responses stream() path then emits Completed, so partial or empty output is reported as authoritative success rather than a terminal failure. Only accept a bare fallback object when it is demonstrably terminal; otherwise return an error for the missing completion event.
AGENTS.md reference: AGENTS.md:L43-L45
Useful? React with 👍 / 👎.
1 participant
Summary
The ChatGPT Codex backend (
chatgpt.com/backend-api/codex) refuses a unary Responses call with HTTP 400{"detail":"Stream must be set to true"}— it only ever speaks SSE. Bothinvokeandstreamroute throughinvoke_responses, which sentstream: None, so every codex-OAuth turn failed before reaching the model. Fixing that surfaced a second defect: with streaming on, the reply came back empty, because Codex ends the stream with aresponse.completedwhoseoutputarray is empty and delivers the content in earlierresponse.output_item.doneevents.Related issue
None.
API or behavior changes
No public API change. Behavior change is scoped to the Responses path and is opt-in by observation:
responses_requires_stream(an internalAtomicBool) latches on the firststream must be set400, retries that one call withstream: true, and sends the flag up front thereafter. A backend that never emits that 400 keeps sending unary requests exactly as before.text/event-stream). Content-type is treated as a hint, not a contract, because Codex omits it on a streamed Responses reply.Not breaking.
Validation
Commands actually run, with their outcome:
cargo fmt --all -- --check— cleancargo clippy --all-targets --all-features -- -D warnings— cleancargo build --all-targets --all-features— okcargo test --all-features— 265 + 7 + 1 passed, 0 failedEnd to end against the live backend, through an embedding host's real chat turn:
gpt-5.5andgpt-5.6-solboth return their reply. Before the change both failed with the 400 above.Tests
Two unit tests in
providers/openai/test.rs:responses_sse_fold_grafts_streamed_output_items_onto_the_completed_response— pins the actual Codex shape (empty terminaloutput, content inresponse.output_item.done) and asserts the parsed text survives. This is the case that produced "The model returned an empty response" in the field.responses_sse_fold_falls_back_to_the_last_seen_response— a stream that never reaches a terminal event still yields the last response seen rather than failing the call.Deliberately untested: the 400-triggered retry itself, which needs a live 400 from the endpoint (no mock server in this crate's test surface). It was exercised against the real backend instead.
Documentation
None needed — the behavior is described in the doc comments on
responses_requires_streamandresponses_sse_final_value, next to the code that implements it. The module note claiming "true SSE streaming remain follow-ups" still holds forstream(), which continues to surface the Responses path as one terminalCompleted.Checklist
#[allow(...)],#[ignore], or relaxed lints.envcontents in the diff or the description