test(harness): a Google ADK reference agent for the hosted path (TH-8809) by khushalsonawat · Pull Request #305 · future-agi/agent-learning-kit

khushalsonawat · 2026-11-11T21:04:13Z

Summary

TH-8809 is part of T43: qualify the hosted harness across frameworks. This is Google ADK's offline half, on the same desk as the OpenAI Agents SDK (#301) and Pydantic AI (#304) references. A local end-to-end run with a real reading and a real model is summarised under Verification.

examples/harness/google_adk_support/ is written the way ADK agents ship: an app package support_agent/ with root_agent, served by ADK's own server (adk web --port 8080 .). The kit already runs that natively: it finds the app, moves the port per world, creates a session per conversation and posts each new message to /run_sse.
- Model: LiteLlm(model="openai/" + MODEL, api_base=OPENAI_BASE_URL, api_key=OPENAI_API_KEY). LiteLLM's OpenAI-compatible provider sends the routed name (vertex_ai/gemini-3.7-flash) unchanged to the model gateway on chat completions. A bare vertex_ai/... LiteLlm name would call Vertex directly, outside the gateway.
- Tools: the same three tools as test(harness): an OpenAI Agents SDK reference agent for the hosted path (TH-8800) #301 and test(harness): a Pydantic AI reference agent for the hosted path (TH-8808) #304, plain functions that ADK calls inline. The kit's tracer records them under support_agent.agent.
- Requirements: google-adk==2.13.0, litellm==1.106.2 and mcp==2.3.0. The pinned LiteLLM imports mcp on the first model call and neither other pin brings it in, so without it a clean install fails that call.
- Sessions: adk web keeps sessions under support_agent/.adk/, which the example's .gitignore excludes.
tests/harness/test_google_adk_reference.py checks:
- Detection: it's detected as google_adk with no blocker, as the run folder, in place and as a subfolder, and a shadowing module changes the result.
- Build: the build recognises the ADK server, picks the app, rewrites the port, derives the session and /run_sse protocol, and binds the tracer to exactly the three tools, each checked against support_agent.agent's real tools.
- Tracer: the real ADK Runner, loaded the way adk web loads it and driven by a stub model, calls the tools, and the real tracer bootstrap records exactly those calls over two turns.
- Model: the model wiring.

No kit code changes. The tests use the shared helpers in tests/harness/reference_agents.py.

Stacked on #301 (feat/th-8800-openai-agents-reference): it uses #301's shared test helpers, so #301 merges first, with a merge commit.

Verification

ruff check on the new files
Focused pytest, run locally on Python 3.12.3: the new file, framework detection, the tracer bootstrap, and the ADK tests in test_bundle_author_v2.py. 326 passed.
Full uv run pytest -q: not run, because no shared runtime is touched.
CI: sdk-smoke runs a fixed list of test files, which didn't include this one. It now does (tests/harness/test_google_adk_reference.py); it runs the kit's own ADK 2.7.1, a kit dependency.
Fix-only checks: the fix made after the review cleared (a no-blockers assert in test_example_in_place_is_not_shadowed_by_the_repository) had a fix-only check: CLEAR, nothing above Low. The mcp pin and the audit fixes (this CI-list line, and this description) had a fix-only check as well: CLEAR, nothing above Low.
Local end-to-end run (12 Nov, a real reading, then the agent on vertex_ai/gemini-3.7-flash through the model gateway): an earlier local step on a stub model had shown a clean install failing the first model call with No module named 'mcp', hence the mcp pin. With it, ADK 2.13.0 gave 3 right replies, with Gemini tool turns through the gateway, and the trace file has the 3 calls. The reading chose a generic http route rather than the kit's native one, and the transcript recorded 5 calls for 3 (see the known limits).
Eleven planted bugs, each caught by its named test:
- the openai/ prefix or api_base dropped;
- an async tool;
- a renamed tool;
- an empty __init__.py;
- a second app package;
- litellm or root_agent removed;
- the folder renamed;
- tools not passed;
- an extra tool.

Review round 1 (Dileep)

The test file uses the shared helpers (from test(harness): an OpenAI Agents SDK reference agent for the hosted path (TH-8800) #301); the detection tests are one parametrised test, with a new shadowing case.
The build test checks each tracer binding against the real support_agent.agent tools, so renaming or dropping a tool fails it.
The assert matching the model in agent.py's source text is gone; the import test checks agent.MODEL.
These had a fix-only check: CLEAR, with no findings (High 0, Medium 0, Low 0). Tests on Python 3.12.3: the ADK reference file (12 tests, from 11), three neighbouring files and the ADK tests in test_bundle_author_v2.py, 317 passed; the planted bugs (a renamed tool, the tracer without its hook) were caught.

Known limits

adk api_server isn't recognised as ADK's server until fix(harness): recognise adk api_server and its agents directory as ADK's HTTP server #259 lands; this example uses adk web.
The tests run the kit's ADK 2.7.1, while the example pins 2.13.0. 2.13.0 ran only in the local end-to-end run.
The kit's ADK protocol takes the reply from the last event's text only.
A reading picks a generic http route for this agent, POST /run on fixed port 8080, not the kit's native route (port per world, /run_sse). Outside this PR; a register row.
ADK's transcript double-records tool calls on that route: each call once more with {} arguments (5 for 3). The runner uses the transcript's calls when there are any, so grading would see the duplicates. This is the kit's tool-evidence parsing, outside this PR; a register row.
On a busy local port, readiness fails: world 0 takes the declared port 8080, and the readiness probe doesn't fall back to IPv4. Local runs only; a register row.
The agent's orders are its own state, not the run's world, as in test(harness): an OpenAI Agents SDK reference agent for the hosted path (TH-8800) #301 (a register row).

Checklist

Public API names stay under agent_learning.*, agent-learn, or @future-agi/agent-learning-kit
New user-facing workflow has docs or an example
New release claim has executable gate coverage (no release claim)
No hosted-service dependency was added to release-gated examples
Security-sensitive output is redacted or justified (no credentials; the key and URL come from the environment)

…809) A small support agent on Google ADK 2.13.0, the same desk as the OpenAI Agents SDK and Pydantic AI references, written the ADK way: an app package with root_agent, served by adk web. Its model goes through the model gateway as LiteLlm openai/<model> with the gateway URL and key, so LiteLLM's OpenAI-compatible provider sends the routed name unchanged on chat completions. Its three tools are plain functions that ADK calls inline. Offline tests pin what the hosted path does with it: it is detected as google_adk; the build recognises the ADK server, app and port and binds the tool tracer to the three tools under support_agent.agent; the real ADK Runner, loaded as adk web loads it and driven by a stub model, makes tool calls that the real tracer records exactly; and the model wiring. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>

A clean install of the reference's requirements failed on the first model call with No module named 'mcp': LiteLLM imports it when a call carries tools, and neither google-adk nor litellm brings it in outside their extras. Pin mcp==2.3.0, which both allow. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>

cdileep23

Please check the comments.

cdileep23 · 2026-11-12T06:34:21Z

+        == ["root_agent"]
+        for node in module.body
+    )
+    assert 'MODEL = os.getenv("OPENAI_MODEL", "vertex_ai/gemini-3.7-flash")' in source


P3. This matches source text, so a reformat of agent.py breaks it, and test_import_needs_no_environment... already checks agent.MODEL against the default. Delete the line.

cdileep23 · 2026-11-12T06:34:21Z

+    assert interface["response_path"] == "content.parts.0.text"
+
+    assert agent.environment["HARNESS_TOOL_TRACE"] == "{{WORLD_DIR}}/agent-tool-calls.jsonl"
+    bindings = json.loads(agent.environment["ALK_TOOL_TRACE_BINDINGS"])


P3. The bindings this asserts are the ones the test writes into the contract itself, and _source_tool_trace_environment copies them through unchanged, so the assert compares the test's input with itself. Renaming return_policy in the example leaves this test green (three other tests catch it).

Fix: assert the bindings against the example's real functions, or drop the bindings claim from the test's name.

cdileep23 · 2026-11-12T06:34:21Z

+DETECTED = {"name": "google_adk", "version": None}
+
+
+def _copy_example(destination: Path) -> Path:


P3. Same copied setup as the other reference-agent PRs (_copy_example, the stand-in writer, the fresh-import fixture, the detection tests, the build test skeleton, the tracer driver, the pin tests). See the shared-helper comment on #301: this file would import from that one module instead.

…dk-reference Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>

…lpers, and check the tracer bindings against the real agent (TH-8809) The test file uses tests/harness/reference_agents.py from #301. The build test checks each tracer binding against support_agent.agent's real tools, so renaming a tool fails it, and the assert matching the model in the agent's source text is gone. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>

cdileep23

Please check the comments.

cdileep23 · 2026-11-13T06:00:23Z

+    name="support_agent",
+    model=LiteLlm(
+        model="openai/" + MODEL,
+        api_base=os.getenv("OPENAI_BASE_URL"),


P2. With OPENAI_API_KEY set and OPENAI_BASE_URL missing or empty, api_base is None and LiteLLM's openai/ provider falls back to https://api.openai.com/v1, so the gateway key is sent to OpenAI. #301 and #304 now refuse this case; this example does not, and README line 25 still documents the fallback.

Repro on the pinned google-adk 2.13.0 and litellm 1.106.2: import support_agent.agent with only OPENAI_API_KEY=gateway-canary-key, block httpx, and call root_agent.model.generate_content_async(...). Three requests go to https://api.openai.com/v1/chat/completions with Authorization: Bearer gateway-canary-key (LiteLLM retries).

Fix: read both variables once at the top of agent.py, raise RuntimeError naming OPENAI_BASE_URL when the key is set and the URL is not, and pass them to LiteLlm. An import with nothing set still works, so the existing tests stay green. Add a test that load_example(OPENAI_API_KEY="k"), and the same with OPENAI_BASE_URL="", raises without the key in the message, and change README line 25 to #301's wording.

cdileep23 · 2026-11-13T06:00:23Z

+    assert sorted(item["name"] for item in bindings) == sorted(
+        tool.__name__ for tool in example.root_agent.tools
+    )
+    for item in bindings:


P3. This binding check is the same loop as #304's build test (test_pydantic_ai_reference.py:226-233), and #301's build test still compares the bindings with its own input. One assert_bindings_match(bindings, module, module_name) in reference_agents.py would cover all three.

khushalsonawat requested a review from cdileep23 November 11, 2026 21:04

This was referenced Nov 11, 2026

test(harness): a Deep Agents reference agent for the hosted path (TH-8810) #306

Open

test(harness): a CrewAI reference agent for the hosted path (TH-8811) #311

Open

khushalsonawat added 2 commits November 12, 2026 10:40

ci: run the Google ADK reference tests in sdk-smoke (TH-8809)

9f7135e

Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>

This was referenced Nov 12, 2026

test(harness): an OpenAI Agents SDK reference agent for the hosted path (TH-8800) #301

Open

test(harness): a Pydantic AI reference agent for the hosted path (TH-8808) #304

Open

cdileep23 requested changes Nov 12, 2026

View reviewed changes

khushalsonawat added 2 commits November 12, 2026 16:07

Merge feat/th-8800-openai-agents-reference into feat/th-8809-google-a…

9f16e2b

…dk-reference Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>

khushalsonawat changed the base branch from feat/TH-8804-source-subfolder to feat/th-8800-openai-agents-reference November 12, 2026 10:41

khushalsonawat requested a review from cdileep23 November 12, 2026 12:06

cdileep23 requested changes Nov 13, 2026

View reviewed changes

Navigation Menu

Repository navigation

test(harness): a Google ADK reference agent for the hosted path (TH-8809) - #305

test(harness): a Google ADK reference agent for the hosted path (TH-8809)#305
khushalsonawat wants to merge 5 commits into
feat/th-8800-openai-agents-referencefrom
feat/th-8809-google-adk-reference

khushalsonawat commented Nov 11, 2026 •

edited

Loading

Uh oh!

cdileep23 left a comment

Uh oh!

cdileep23 Nov 12, 2026

Uh oh!

cdileep23 Nov 12, 2026

Uh oh!

cdileep23 Nov 12, 2026

Uh oh!

cdileep23 left a comment

Uh oh!

cdileep23 Nov 13, 2026

Uh oh!

cdileep23 Nov 13, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		DETECTED = {"name": "google_adk", "version": None}


		def _copy_example(destination: Path) -> Path:

Conversation

khushalsonawat commented Nov 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Verification

Review round 1 (Dileep)

Known limits

Checklist

Uh oh!

cdileep23 left a comment

Choose a reason for hiding this comment

Uh oh!

cdileep23 Nov 12, 2026

Choose a reason for hiding this comment

Uh oh!

cdileep23 Nov 12, 2026

Choose a reason for hiding this comment

Uh oh!

cdileep23 Nov 12, 2026

Choose a reason for hiding this comment

Uh oh!

cdileep23 left a comment

Choose a reason for hiding this comment

Uh oh!

cdileep23 Nov 13, 2026

Choose a reason for hiding this comment

Uh oh!

cdileep23 Nov 13, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

khushalsonawat commented Nov 11, 2026 •

edited

Loading