Repository navigation
fix(harness): let an OpenAI Agents SDK agent run on the model gateway (TH-8800) - #303
khushalsonawat wants to merge 2 commits into
Conversation
For a target model routed through the gateway (it contains / and its first segment is not openai, e.g. vertex_ai/gemini-3.7-flash), an Agents SDK agent failed on its first turn: the SDK refuses an unknown name prefix, and it sends the Responses API, which the gateway serves only for its OpenAI provider. The authoring and repair guidance now tells the setup to give such an agent the model as openai/<target_model>, which the SDK strips once. The build marks the agent's control process, and the kit's agent sitecustomize then sets the SDK to chat completions through its public setting after the agent's own import of the SDK. That happens only with the mark and with openai-agents installed. It never imports the SDK itself, never changes a shared loader, lets the SDK's own import errors through, and leaves the agent's own later choice in force. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…ke (TH-8800) Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
| environment = {**declared_runtime_environment, **environment} | ||
| if routed_through_gateway(job.agent.config.get("target_model")): | ||
| # Read at start-up by the shared sitecustomize hook. | ||
| environment["ALK_OPENAI_AGENTS_CHAT_COMPLETIONS"] = "1" |
There was a problem hiding this comment.
P2. This makes an unchanged Agents SDK agent run on the gateway, but the SDK's own trace export stays on. On openai-agents 0.24.0 it posts spans to https://api.openai.com/v1/traces/ingest with os.environ["OPENAI_API_KEY"] (agents/tracing/processors.py), and on a gateway run that key is the gateway's.
Failure: every gateway run of a customer's SDK agent sends the gateway key and the agent's spans to OpenAI. #301's example turns tracing off in its own code; a customer's agent won't.
Repro: build a gateway job and read the control service's environment. It has ALK_CALLBACK_ENTRYPOINT, ALK_OPENAI_AGENTS_CHAT_COMPLETIONS, HARNESS_MODE and PORT, and no OPENAI_AGENTS_DISABLE_TRACING.
Fix: set environment["OPENAI_AGENTS_DISABLE_TRACING"] = "1" next to the mark here and at :2866 (the SDK reads it in agents/tracing/provider.py), with a test on both paths.
| ) | ||
| if routed_through_gateway(target_model): | ||
| instructions.append( | ||
| "If the agent uses the OpenAI Agents SDK and takes its model as a name, give it " |
There was a problem hiding this comment.
P2. "takes its model as a name" also matches an agent that reads a name and hands it to OpenAIChatCompletionsModel(model=...), like #301's example. That class sends the name as written; only the default provider strips openai/.
Failure: the setup follows this sentence, sets OPENAI_MODEL=openai/vertex_ai/gemini-3.7-flash, and the gateway receives "model": "openai/vertex_ai/gemini-3.7-flash" on every request.
Repro: run #301's agent on openai-agents 0.24.0 against a local stand-in gateway with that OPENAI_MODEL and log the request body.
Fix: limit the sentence to a name passed as Agent(model=...) or RunConfig(model=...), and say a name handed to a model object stays unchanged.
| code = ( | ||
| "import agents, site, sys\n" | ||
| + _note_line("before-bootstrap") | ||
| + f"site.addsitedir({str(hook)!r})\n" |
There was a problem hiding this comment.
P2. site.addsitedir appends the hook folder after the standard library, and import sitecustomize loads the first one on sys.path. On a Python that ships its own sitecustomize.py (Homebrew 3.12.11 has one in lib/python3.12/), this test loads that file instead of the kit's.
Failure: At index 2 diff: 'after-bootstrap' != 'chat_completions' on Homebrew Python. CI passes only because setup-python builds ship no such file.
Fix: sys.path.insert(0, hook) in the driver. The test passes with that change.
| livekit_hook_due = ( | ||
| "HARNESS_TOOL_TRACE" in process.environment | ||
| or "FI_WORKER_HEALTH_PORT" in process.environment | ||
| or "ALK_OPENAI_AGENTS_CHAT_COMPLETIONS" in process.environment |
There was a problem hiding this comment.
P3. The mark is set for every gateway-routed target model (bundle_author_v2.py:2596 and :2866), not only for Agents SDK agents, so every Python agent on a gateway run now gets the kit's sitecustomize.py. That hides a repository's own sitecustomize and doesn't hand on to it. The description lists the shadowing as a known limit, but it now reaches LangGraph, CrewAI and the rest, which gain nothing from the hook.
Repro: a repo with its own sitecustomize.py that prints repo hook ran, built as a non-SDK agent on a gateway job: nothing is printed.
Fix: set the mark only when framework detection says openai_agents, or chain to the next sitecustomize.
| import importlib.metadata | ||
|
|
||
| importlib.metadata.distribution("openai-agents") | ||
| except Exception: # noqa: BLE001 - no SDK installed: nothing to switch |
There was a problem hiding this comment.
P3. A few small things in this hook:
- here,
except Exceptionaroundimportlib.metadata.distributionshould beexcept importlib.metadata.PackageNotFoundError. - the
except Exception: passaround_install_openai_agents_chat_style()below writes nothing, unlike the mark hook's catch at the top of the file. Nothing in that function raises past its own catches, so drop it or write onealk:line. _set_openai_agents_chat_styleraises aRuntimeErroronly to catch it on the next line; write the stderr line directly.- the comment above ("a customer's own package called
agentsis never touched") claims more than the check does: it proves the distribution is installed, not that the importedagentscame from it.test_openai_agents_chat_style_failure_is_one_stderr_line_and_the_import_succeedswraps a non-SDKagents. Reword it.
Summary
When a job asks for a model routed through the platform's model gateway (a
target_modelsuch asvertex_ai/gemini-3.7-flash), an OpenAI Agents SDK agent fails on its first turn in two ways:Unknown prefix: vertex_ai)./v1/responsesonly for its OpenAI provider. Every other provider returns 501.This change makes such an agent work on the gateway without touching its code:
/and its first segment isn'topenai), the authoring and repair guidance tells the agent's setup to give an Agents SDK agent the model asopenai/<target_model>. The SDK strips exactly oneopenai/and sends the rest unchanged to its configured endpoint.ALK_OPENAI_AGENTS_CHAT_COMPLETIONS=1(native processes and the Compose control service). The kit's existing agent-processsitecustomizethen sets the SDK to chat completions through its publicset_default_openai_api("chat_completions"), after the agent's ownimport agentshas run. Why: the gateway can't serve the SDK's default style for non-OpenAI models.openai-agentsdistribution isn't installed.set_default_openai_api, still wins.alk:line goes to stderr and the agent runs as it would without the kit.openai/...names are left alone, so direct OpenAI runs keep the Responses API, hosted tools,previous_response_idandprompt=.Overlaps with open PRs that touch the same files: #168 (
job.py,process_runtime.py), #270, #255, #262, #258, #257, #226, #209 and #204. Each file's change here is small: one helper and one sentence, two environment entries, one condition, one hook.Verification
ruff checkon the changed filesuv run pytest -q: the four touched modules were run whole insteadsdk-smokeruns a fixed list of test files.test_bundle_author_v2.pyandtest_process_runtime.pywere on it;test_python_tool_trace_bootstrap.pyandtest_workflow_intent.pyweren't, and now are. Both pass whole on the base. In CI, one test skips:test_python_tool_trace_bootstrap.py:338, the real-SDK pin, because CI doesn't installopenai-agents. The audit fixes (this CI-list line, and this description) had a fix-only check as well: CLEAR, nothing above Low.vertex_ai/gemini-3.7-flashthrough the model gateway): this PR's model-name path held, and the run used chat completions, with no Responses call.openai-agents. It skips where the SDK isn't installed; its names were checked against the 0.24.0 source. It hasn't been run with the SDK installed yet: installing it in a test environment is waiting for a go.Known limits
target_modeltoday, so this applies to jobs whose callers send one (a register row).openai-agentsis installed in the test interpreter; no CI job installs it.vertex_ai/...with Google credentials) is told to useopenai/..., wherelitellm/...would work. That is for a decision.sitecustomize.py, which doesn't chain to a repository's ownsitecustomize. This is already true for traced and LiveKit processes.litellm/...decision) (CLEAR: no High; cost about 40 ns per import, no measurable change across 343 modules; 8 concurrent importers gave one setting):RecursionErrorin the two-copy case. No real trigger was found (wrapt guards itself). Fix if it ever appears: a thread-local re-entrancy guard infind_spec.copyorpickleof the SDK's top-level loader recurses.isinstance(agents.__loader__, SourceFileLoader)is False, sopkg_resourcesuses its generic provider for the top-level package only.Checklist
agent_learning.*,agent-learn, or@future-agi/agent-learning-kit