Skip to content

fix(harness): let an OpenAI Agents SDK agent run on the model gateway (TH-8800) - #303

Open
khushalsonawat wants to merge 2 commits into
feat/TH-8804-source-subfolderfrom
fix/th-8800-agents-sdk-on-gateway
Open

khushalsonawat wants to merge 2 commits into
feat/TH-8804-source-subfolderfrom
fix/th-8800-agents-sdk-on-gateway

Conversation

@khushalsonawat

@khushalsonawat khushalsonawat commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Summary

When a job asks for a model routed through the platform's model gateway (a target_model such as vertex_ai/gemini-3.7-flash), an OpenAI Agents SDK agent fails on its first turn in two ways:

  • The name. The SDK refuses a model name whose first segment it doesn't know (Unknown prefix: vertex_ai).
  • The request style. The SDK sends OpenAI's Responses API by default, and the gateway serves /v1/responses only for its OpenAI provider. Every other provider returns 501.

This change makes such an agent work on the gateway without touching its code:

  • The guidance. When the target model is routed (it contains / and its first segment isn't openai), the authoring and repair guidance tells the agent's setup to give an Agents SDK agent the model as openai/<target_model>. The SDK strips exactly one openai/ and sends the rest unchanged to its configured endpoint.
  • The kit sets the SDK's request style on gateway runs. For a routed target model, the build marks the agent's control process with ALK_OPENAI_AGENTS_CHAT_COMPLETIONS=1 (native processes and the Compose control service). The kit's existing agent-process sitecustomize then sets the SDK to chat completions through its public set_default_openai_api("chat_completions"), after the agent's own import agents has run. Why: the gateway can't serve the SDK's default style for non-OpenAI models.
    • It does nothing without the mark, or when the openai-agents distribution isn't installed.
    • It never imports the SDK itself, and it never changes a loader another module shares.
    • The SDK's own import errors reach the agent unchanged, and the setting is made once.
    • The agent's own later choice, such as an explicit model object or its own set_default_openai_api, still wins.
    • If the setting can't be made, one alk: line goes to stderr and the agent runs as it would without the kit.
  • Bare names and openai/... names are left alone, so direct OpenAI runs keep the Responses API, hosted tools, previous_response_id and prompt=.

Overlaps with open PRs that touch the same files: #168 (job.py, process_runtime.py), #270, #255, #262, #258, #257, #226, #209 and #204. Each file's change here is small: one helper and one sentence, two environment entries, one condition, one hook.

Verification

  • ruff check on the changed files
  • Focused pytest, run locally on Python 3.12.3 (see below)
  • Full uv run pytest -q: the four touched modules were run whole instead
  • CI: sdk-smoke runs a fixed list of test files. test_bundle_author_v2.py and test_process_runtime.py were on it; test_python_tool_trace_bootstrap.py and test_workflow_intent.py weren't, and now are. Both pass whole on the base. In CI, one test skips: test_python_tool_trace_bootstrap.py:338, the real-SDK pin, because CI doesn't install openai-agents. The audit fixes (this CI-list line, and this description) had a fix-only check as well: CLEAR, nothing above Low.
  • Local end-to-end run (12 Nov, with test(harness): an OpenAI Agents SDK reference agent for the hosted path (TH-8800) #301's agent on SDK 0.24.0 and vertex_ai/gemini-3.7-flash through the model gateway): this PR's model-name path held, and the run used chat completions, with no Responses call.
  • Planted bugs, each caught by a named test:
    • the 10 from the plan and a duplicated guidance sentence (the review: 11 of 11);
    • after the review's fix, the finder never disarmed, and disarmed on lookup.
  • A real-SDK pin runs the hook against an installed openai-agents. It skips where the SDK isn't installed; its names were checked against the 0.24.0 source. It hasn't been run with the SDK installed yet: installing it in a test environment is waiting for a go.

Known limits

  • The platform doesn't set target_model today, so this applies to jobs whose callers send one (a register row).
  • A bare non-OpenAI name (no prefix) stays on Responses and gets the gateway's 501.
  • The real-SDK pin skips unless openai-agents is installed in the test interpreter; no CI job installs it.
  • A direct run on another provider's own keys (vertex_ai/... with Google credentials) is told to use openai/..., where litellm/... would work. That is for a decision.
  • Gateway-run processes now get the kit's sitecustomize.py, which doesn't chain to a repository's own sitecustomize. This is already true for traced and LiveKit processes.
  • The gateway gap itself (Responses only on the OpenAI provider, and no provider-prefix stripping on Responses) is a High row in the older-findings register, for a gateway change.
  • From a review of the import-hook change made after the main review cleared (one register row covers these and the litellm/... decision) (CLEAR: no High; cost about 40 ns per import, no measurable change across 343 modules; 8 concurrent importers gave one setting):
    • Re-entrancy: if another meta-path finder that walks the finder list without a guard sits ahead of the hook, or two copies of the hook are loaded, the two finders call each other. The import still succeeds with nested loader wrappers, or fails with RecursionError in the two-copy case. No real trigger was found (wrapt guards itself). Fix if it ever appears: a thread-local re-entrancy guard in find_spec.
    • Not changing a shared (zip) loader isn't pinned by a test.
    • copy or pickle of the SDK's top-level loader recurses.
    • isinstance(agents.__loader__, SourceFileLoader) is False, so pkg_resources uses its generic provider for the top-level package only.

Checklist

  • Public API names stay under agent_learning.*, agent-learn, or @future-agi/agent-learning-kit
  • New user-facing workflow has docs or an example (no new workflow)
  • New release claim has executable gate coverage (no release claim)
  • No hosted-service dependency was added to release-gated examples
  • Security-sensitive output is redacted or justified (the stderr line names an exception type only)

For a target model routed through the gateway (it contains / and its first
segment is not openai, e.g. vertex_ai/gemini-3.7-flash), an Agents SDK agent
failed on its first turn: the SDK refuses an unknown name prefix, and it
sends the Responses API, which the gateway serves only for its OpenAI
provider.

The authoring and repair guidance now tells the setup to give such an agent
the model as openai/<target_model>, which the SDK strips once. The build
marks the agent's control process, and the kit's agent sitecustomize then
sets the SDK to chat completions through its public setting after the
agent's own import of the SDK. That happens only with the mark and with
openai-agents installed. It never imports the SDK itself, never changes a
shared loader, lets the SDK's own import errors through, and leaves the
agent's own later choice in force.

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…ke (TH-8800)

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>

@cdileep23 cdileep23 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check the comments.

environment = {**declared_runtime_environment, **environment}
if routed_through_gateway(job.agent.config.get("target_model")):
# Read at start-up by the shared sitecustomize hook.
environment["ALK_OPENAI_AGENTS_CHAT_COMPLETIONS"] = "1"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2. This makes an unchanged Agents SDK agent run on the gateway, but the SDK's own trace export stays on. On openai-agents 0.24.0 it posts spans to https://api.openai.com/v1/traces/ingest with os.environ["OPENAI_API_KEY"] (agents/tracing/processors.py), and on a gateway run that key is the gateway's.

Failure: every gateway run of a customer's SDK agent sends the gateway key and the agent's spans to OpenAI. #301's example turns tracing off in its own code; a customer's agent won't.

Repro: build a gateway job and read the control service's environment. It has ALK_CALLBACK_ENTRYPOINT, ALK_OPENAI_AGENTS_CHAT_COMPLETIONS, HARNESS_MODE and PORT, and no OPENAI_AGENTS_DISABLE_TRACING.

Fix: set environment["OPENAI_AGENTS_DISABLE_TRACING"] = "1" next to the mark here and at :2866 (the SDK reads it in agents/tracing/provider.py), with a test on both paths.

Comment thread src/fi/alk/harness/job.py
)
if routed_through_gateway(target_model):
instructions.append(
"If the agent uses the OpenAI Agents SDK and takes its model as a name, give it "

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2. "takes its model as a name" also matches an agent that reads a name and hands it to OpenAIChatCompletionsModel(model=...), like #301's example. That class sends the name as written; only the default provider strips openai/.

Failure: the setup follows this sentence, sets OPENAI_MODEL=openai/vertex_ai/gemini-3.7-flash, and the gateway receives "model": "openai/vertex_ai/gemini-3.7-flash" on every request.

Repro: run #301's agent on openai-agents 0.24.0 against a local stand-in gateway with that OPENAI_MODEL and log the request body.

Fix: limit the sentence to a name passed as Agent(model=...) or RunConfig(model=...), and say a name handed to a model object stays unchanged.

code = (
"import agents, site, sys\n"
+ _note_line("before-bootstrap")
+ f"site.addsitedir({str(hook)!r})\n"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2. site.addsitedir appends the hook folder after the standard library, and import sitecustomize loads the first one on sys.path. On a Python that ships its own sitecustomize.py (Homebrew 3.12.11 has one in lib/python3.12/), this test loads that file instead of the kit's.

Failure: At index 2 diff: 'after-bootstrap' != 'chat_completions' on Homebrew Python. CI passes only because setup-python builds ship no such file.

Fix: sys.path.insert(0, hook) in the driver. The test passes with that change.

livekit_hook_due = (
"HARNESS_TOOL_TRACE" in process.environment
or "FI_WORKER_HEALTH_PORT" in process.environment
or "ALK_OPENAI_AGENTS_CHAT_COMPLETIONS" in process.environment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3. The mark is set for every gateway-routed target model (bundle_author_v2.py:2596 and :2866), not only for Agents SDK agents, so every Python agent on a gateway run now gets the kit's sitecustomize.py. That hides a repository's own sitecustomize and doesn't hand on to it. The description lists the shadowing as a known limit, but it now reaches LangGraph, CrewAI and the rest, which gain nothing from the hook.

Repro: a repo with its own sitecustomize.py that prints repo hook ran, built as a non-SDK agent on a gateway job: nothing is printed.

Fix: set the mark only when framework detection says openai_agents, or chain to the next sitecustomize.

import importlib.metadata

importlib.metadata.distribution("openai-agents")
except Exception: # noqa: BLE001 - no SDK installed: nothing to switch

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3. A few small things in this hook:

  • here, except Exception around importlib.metadata.distribution should be except importlib.metadata.PackageNotFoundError.
  • the except Exception: pass around _install_openai_agents_chat_style() below writes nothing, unlike the mark hook's catch at the top of the file. Nothing in that function raises past its own catches, so drop it or write one alk: line.
  • _set_openai_agents_chat_style raises a RuntimeError only to catch it on the next line; write the stderr line directly.
  • the comment above ("a customer's own package called agents is never touched") claims more than the check does: it proves the distribution is installed, not that the imported agents came from it. test_openai_agents_chat_style_failure_is_one_stderr_line_and_the_import_succeeds wraps a non-SDK agents. Reword it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants