[khushalsonawat]

Summary

TH-8800 is a proof of concept: take one OpenAI Agents SDK agent through the hosted flow (reading, build, simulation, graded run) and list what breaks, before five more frameworks follow. This PR is the agent and the offline half. A local end-to-end run with a real reading and a real model is summarised under Verification.

• examples/harness/openai_agents_support/ is a small support agent, written the way a customer would:
  • three plain function tools over a small in-memory order list;
  • a module-level agent_callback for the kit's callback bridge;
  • the model built per call as a chat-completions model object;
  • the SDK's own trace export turned off.
• The model object matters: the SDK's default provider refuses a gateway name such as vertex_ai/....
• The agent returns content only. The SDK runs its tools, and the kit's tool tracer records each call.
• tests/harness/test_openai_agents_reference.py pins what the hosted path does with the agent. It uses a stand-in for the SDK, since the SDK isn't a kit dependency:
  • it's detected as openai_agents, including as a subfolder of a tree with a nested agents/ folder;
  • the build runs it through the callback bridge, with the tool tracer bound to exactly its three tools;
  • the real tracer bootstrap records each tool as 0.24.0 calls it (asyncio.to_thread from a request thread), and nothing for a same-named function in another module;
  • the callback sends the conversation once, builds and closes a client per call, and replies with content only;
  • the requirements are pinned.
• tests/harness/reference_agents.py holds the helpers the reference agent tests share: copying an example, writing stand-ins, a fresh import, the detection cases (run folder, subfolder, in place, and a shadowing module), the build, the tracer driver and the pin check. It is the only module importing test_bundle_author_v2's private names. The Pydantic AI, Google ADK, Deep Agents and CrewAI references (test(harness): a Pydantic AI reference agent for the hosted path (TH-8808) #304, test(harness): a Google ADK reference agent for the hosted path (TH-8809) #305, test(harness): a Deep Agents reference agent for the hosted path (TH-8810) #306, test(harness): a CrewAI reference agent for the hosted path (TH-8811) #311) stack on this PR and use it, so this PR merges first, with a merge commit.
• The example refuses a key with no base URL: if OPENAI_API_KEY is set and OPENAI_BASE_URL isn't, the callback raises before building a client, naming the missing setting.

No kit code changes.

Verification

• ruff check on the new files
• Focused pytest, run locally on Python 3.12.3: the new file, plus test_framework_detection.py, test_python_tool_trace_bootstrap.py and test_callback_http_adapter.py. 291 passed.
• Full uv run pytest -q: not run, because no shared runtime is touched.
• CI: sdk-smoke runs a fixed list of test files, which didn't include this one. It now does (tests/harness/test_openai_agents_reference.py).
• Fix-only checks: the fixes made after the review cleared (the tracer test's decoy now differs only by module, two comments, a README line) had a fix-only check: CLEAR, nothing above Low, and its plant (the module check always true) was caught. The audit fixes (this CI-list line, and this description) had a fix-only check as well: CLEAR, nothing above Low.
• Local end-to-end run (12 Nov, a real reading, then the agent on vertex_ai/gemini-3.7-flash through the model gateway): the agent gave the right replies on chat completions. The reading saved its tools as unreachable, with the module from the repo root, so no calls were recorded. An earlier local run with a hand-written import entry from module agent recorded all 3.
• Ten planted bugs, each caught by its named test, plus one for the tracer's module check. These include returning tool calls, a bare model string, a client built at import, tracing left on, the new message sent twice, an async tool and a missing requirement.

Review round 1 (Dileep)

• The shared test helpers (tests/harness/reference_agents.py), with a parametrised detection test, and this file moved onto them.
• A README line says agent-simulate is pinned because the kit's callback bridge imports fi.simulate.AgentInput.
• The example refuses a key given with no base URL, before building a client.
• These had a fix-only check: CLEAR, no findings (3 plants caught: the refusal off, the hook off the path, no shadow planted). Tests: the four files above, 294 passed on Python 3.12.3.
• CI at 5d45834: the pull_request run of clean-install failed once on tests/harness/test_contract_model.py::test_a_crafted_file_of_block_comments_does_not_hold_up_the_check[a-file-of-openers], a timing test in a file this PR doesn't touch (0.525 s against 0.5 s): https://github.com/future-agi/agent-learning-kit/actions/runs/43196563653. The push run of the same commit is green (https://github.com/future-agi/agent-learning-kit/actions/runs/43196557257); the failed job was re-run.

Known limits

• The tests use a stand-in for the SDK, because CI doesn't install it. SDK 0.24.0 itself ran only in the local end-to-end run.
• A reading doesn't enter @function_tool tools in a way the tracer can bind. It saves them as unreachable, with the module from the repo root, so a hosted run of this agent gets no tool evidence today. This is the reading's guidance, outside this PR, and it's a High row in the older-findings register.
• The agent's orders are its own state, not the run's world, so checks that read world state can't see a return it starts (a register row).
• Renaming the example folder to agents doesn't block detection; only a root-level agents/ or agents.py does. The test pins the folder name instead.

Checklist

• Public API names stay under agent_learning.*, agent-learn, or @future-agi/agent-learning-kit
• New user-facing workflow has docs or an example
• New release claim has executable gate coverage (no release claim)
• No hosted-service dependency was added to release-gated examples
• Security-sensitive output is redacted or justified (no credentials; the key and URL come from the environment)

[cdileep23]

Please check the comments.

[cdileep23]

P3. _copy_example, the stand-in writer, the fresh-import fixture, the three detection tests, the build test skeleton, the tracer subprocess driver and the pin tests are repeated nearly line for line in #304, #305, #306 and #311 (#304's test file shares 339 lines with this one). All five also import the private _authoring and _github_job from test_bundle_author_v2.

Fix: this is the first of the series, so land one tests/harness/reference_agents.py here with those helpers and a parametrised detection test, and have the others import it. The copied _ORDERS and tools inside each example folder are fine, since each folder has to stand alone.

[cdileep23]

P3. Nothing in agent.py imports agent-simulate, so a reader will take this pin as unused. It's needed because the kit's callback bridge imports fi.simulate.AgentInput. One line in the README saying so would do.

[cdileep23]

Please check the comments.

[cdileep23]

P3. pytest only rewrites asserts in files matching python_files = ["test_*.py"] and in conftests, and nothing registers this module, so a failing pin or detection check in any of the five reference test files prints a bare AssertionError with no values.

Repro: change one pin in requirements.txt and run test_requirements_and_python_version_are_pinned.

Fix: pytest.register_assert_rewrite("reference_agents") in tests/harness/conftest.py.

[cdileep23]

P3. This passes when detection still finds the framework but with a blocker, so it is looser than the test it replaced (#311's old check was detect_framework(run) != CREWAI). Today a shadowed import is skipped outright (framework_detection.py:486), so nothing hides yet, but a change that reports the framework with a blocker would stay green.

Fix: assert found != expected in the shadowed branch.

[cdileep23]

P3. The PR body says the Deep Agents reference (#306) stacks on this PR and uses the helper. At #306's head (3f68c5e) it still has its own _copy_example, _write_stand_in and _build (test_deep_agents_reference.py:20, 119, 124, 222). Either move #306 onto it or reword the body.