[khushalsonawat]

Based on dev. #224 (TH-8806) is merged into dev, and dev is merged into this branch (c07f4bb, a merge), so the diff shows only TH-8804. TH-8805 (#229) was merged into this branch and comes with it; see "TH-8805 is included" below.

Summary

One repo can hold several agents. A job can now name one with source.subfolder, and the kit treats that folder as the agent, with the whole repo still present so a project file above it can be used. The kit runs the agent from the folder and sends the value back as extension.source.subfolder. A folder it can't use fails with subfolder_not_found (domain agent, stage acquiring_source, not retried) before any model call. With no folder, or an empty one, builds are byte-identical to today, except where the save's read limits below apply.

Ticket: TH-8804.

Release order

HARNESS_SOURCE_SUBFOLDER_ENABLED stays off until this is out:

1. Platform #3661 ships with the setting off.
2. The kit release with TH-8806 (feat(harness): refuse browser, computer-use and crawler agents #224) and this change is deployed on the hosted template, the sandbox server and the Compose broker. The broker selects the runtime again, so an older one would drop the field and pick the root Compose file.
3. The setting goes on.

Known limits

• Sandbox server route: with #3661, _flatten_source sets generic_harness_v1 when a subfolder is sent, so the route's refusal no longer applies to platform jobs. That route sends no echo, and it returns 500 for every GitHub job on dev: TH-8978.
• Sibling parts can still be sealed if the model names them. Not guarded: data_store.schema_from, system_prompt_from, requirements_files. Not parsed: command forms such as python -c, env -C../b, uv run -d../b, gunicorn --pythonpath=../b,. and env PYTHONPATH=../b:..
• Test-only re-runs never compare their bundle's folder with the job's, and send no document. Whether that case can happen isn't proven.
• A folder deleted after building: an "add scenarios" run spends model money before bundling refuses it. A proper fix needs the platform to run the check first.
• Own project in the folder, shared code above it: an agent with its own project ships only its folder, so a path dependency above the folder won't install. It fails loudly at install, as the same layout does today.
• Any Compose fault other than a service that builds another agent's folder, on a folder build: the save lets it through, as with no folder, and the build meets it later. On a fresh build that isn't at once: some faults end the builder conversation, and the build waits out its hour, then shows as a crash the platform may retry; others fail only after a full build, with the cause shown and no retry. A mistake the model made in its own Compose settings, such as a typo in the file name, is no longer reported at the save, so it costs the build rather than a turn. The same goes for a bad alk.yaml.
• Odd files can crash bundling with no failure file, with or without a folder, as they do on dev today: a separate ticket.
• The import walk has budgets: 2,000 files, 32 MB and 20,000 import names; at the save, also 5,000 import names, 100,000 lookups on disk (a file found costs one for each part of its path, a folder link one for each part of its target), a 256 KB parse total and a 2 MiB script total. It reads the folder's own files first, and doesn't enter .git, .venv, node_modules or the other folders no one's code is in. A Python file with no import in it doesn't use up the parse total. Past a budget the save's walk logs that it stopped and the credentials manifest says truncated; anything it didn't reach counts as not the agent's. Instrumentation is still recorded from what the walk reached, so a stopped walk can record none: TH-9509.
• Imports the walk can't see: sys.path edits and importlib, and JavaScript or TypeScript packages only a workspace or path alias can resolve. A uv workspace is followed as the build's uv sync --no-dev --package installs it: the agent project's dependencies, all its extras and its default groups other than dev (with include-group), and, from each workspace member or in-repository path source it reaches, what that one installs, with the extras asked of it, the root's sources applying to every member. At most 32 members are followed and 256 read, and 1 MiB of pyproject.toml in all; a ** member pattern isn't expanded, and past a limit the walk says it stopped. A plan that asks for an extra a member declares makes the build sync every member, which the walk doesn't follow: TH-9513. A folder link is followed to its target inside the repository, charged a lookup for each part, and a path or link target past 128 parts below the root isn't; a link to a file isn't followed, a folder under an ignored name such as vendor/ doesn't find its own keys, and a multi-line import type is followed as if real. When the walk can't tell whose key something is, it says optional, so a missing key fails at the agent's first call. Nothing on the hosted path reads this list yet.
• A .py is read in the encoding its coding line names, as Python reads it, except a codec that isn't text (such as rot13), punycode, whose decode costs the square of its size, idna, whose decode runs in Python for each label, and undefined. Those files are skipped by the scans (the browser-tool check still counts the browser packages and tool markers they name, as whole names in their bytes, since Python runs them; feat(harness): refuse browser, computer-use and crawler agents #224 does the same). A unicode-escape file is decoded as Python decodes it, so an import spelled in escapes counts, up to 4 MiB per scan; past that it is searched as bytes. Three rarer codecs can still spell an import in bytes the scans don't see (raw_unicode_escape, utf-7, iso2022_jp; only deliberate hiding reaches it): TH-9490. A non-ASCII byte on the coding line is read as Python reads it. A first line that holds more than 64 places that look like a cookie but aren't, before the real one, is read as a refused codec's file is. The import walk reads all of these, unicode-escape too, as UTF-8, as it does a file its codec can't decode (which Python refuses too): decoding with replacement runs Python's own error handler for each byte in most codecs. So a crafted file Python reads in idna or punycode can give the walk fewer imports than Python finds. Python's own refusals of a non-UTF-8 byte on the coding line's own line, and of utf-16 or utf-32 named on it, aren't copied: such a file is read anyway.
• Names are matched as written: a fullwidth ｐlaywright or ｔraceai_openai, which Python reads as the plain name, isn't found by the save's checks. A .py holding a run of 32 combining marks isn't parsed, and a ${} inside a JavaScript template literal isn't followed.
• The import walk follows nothing more than 64 folders below the repository.
• A shared helper outside the folder, such as a root infra/ service the stack runs, is refused as building another agent's folder; it can be pulled as an image instead.
• Files directly above the folder count as this agent's, so a root pyproject.toml naming a provider can require its keys on an agent that doesn't use it.
• A tool importing a sibling's module by flat name won't find it. Only the repo root and the agent folder are on the import path.
• A sibling's credential-like file can refuse this build when the install root is the repo root, as it does for a root build today.

Differences from the approved design

• The description records the folder (runtime.source_subfolder) on subfolder builds only, so it also appears inside extension.contract.
• The builder session has no folder check of its own. Until TH-8806's guard is in, it can take a model turn on a job that has already failed.
• An empty folder counts as not found. A reused build from another folder fails as subfolder_not_found, naming both.
• A link to elsewhere in the repo is accepted, not refused.
• An agent with its own project below the subfolder runs from that project, after one repair turn. On certify-only, which can't repair, the build ends.
• The start command, workspaces, callback and graph entries are relative to the working folder, not the repo root.
• The hosted run reports a bad folder through its terminal event, because the failure file isn't read on exit 0.
• With a folder set, the kit refuses a shell as the start command, a command or Compose stack outside the folder, and any Compose service in the stack, or a Dockerfile, that builds a sibling; the refusal names the service. An explicit root compose.yaml is allowed only when every service it runs builds the folder or above it. With no service named, a service behind a profile isn't checked unless COMPOSE_PROFILES asks for one of its profiles, since nothing else starts it.
• Seed data and new data rules read only the folder and those above it. Voice tool certification reads the folder or the nearest project at or above it.
• A selection has one budget for the whole Compose file. Compose files and alk.yaml refuse YAML of more than 200,000 nodes, counted across every included file (an alias repeats a node without counting it, so network_mode, ipc and pid must be strings and environment values plain, as Compose takes them), << merges nested more than 32 deep, [ or { nested more than 16 deep, a %TAG prefix over 1,024 characters, a string that interpolation would build past 64 KiB in all, counting what it keeps and what it adds (compose_structural_interpolation_limit), too many lookups on disk to resolve the stack's build, include, env-file and mount paths, a link's target counted, and each empty, . or .. part walked (the same code), a path over 4,096 characters (compose_source_path_invalid, compose_include_path_invalid), and a base-60 number (1:2:3) of more than 17 parts, whose cost grows with the square of its length. At a Compose file the code names the reason (compose_yaml_too_large, compose_yaml_too_deep, compose_yaml_tag_too_long, …); at alk.yaml it follows runtime_manifest_invalid:. alk.yaml over 1 MiB is refused, read only that far.
• The save's checks run off the event loop. The contract is written only after them, unless the agent was refused meanwhile. When two contracts in one message race, the one sent last is saved.
• The save reads less Python, with or without a folder, because Python's parser holds the builder conversation while it runs. On 3.12, one crafted 1 MiB .py held the conversation 13.5 s per parse: a folder save took 55 s, and a save without a folder 40 s, as on base (41 s). Now a .py is parsed only if it's a regular file of at most 128 KB. Each scan parses at most 256 KB, and tokenizes at most 64 KB, counted in bytes; each token is also charged the line it's read from, since 3.12.3, the sandbox's Python, decodes the line again for each token (one 64 KB line of short tokens took 3.8 s). A file holding a run of 32 combining marks isn't parsed: on 3.11 and 3.12.3 the parser takes seconds to fold such a name. The instrumentation and browser-tool scans parse only files that name something they look for. On 40 crafted 128 KB files, a save takes about 1.6 s on 3.12 (4.0 s on 3.11), and the conversation pauses at most 0.27 s (0.8 s on 3.11). Imports in a .py over 128 KB, or past a scan's total, don't count, and past the tokenize total the browser-tool scan doesn't read a file's markers. At the save, the import walk matches at most 2 MB of scripts and tries at most 5,000 import names, and with or without a save it follows nothing more than 64 folders below the repository, looks up each package and each file it finds once, keeps no other misses, and doesn't try a name over 4,096 characters; before, a crafted TypeScript file held the conversation about 20 s, and a chain of 358 nested packages made a save wait a minute. The browser-tool check reads at most 16 MB of scripts, with or without a folder; before, 128 MB of crafted scripts made a save wait 14.6 s (base code), and in a monorepo saved without a folder, scripts past 16 MB now aren't checked for browser tools. Before, a link to /dev/zero named .py held every save (base code).
• The save's check for a service that builds another agent's folder reads at most 128 KB of alk.yaml and Compose files together (the largest real Compose file seen is 53 KB), and skips a runtime with Compose overlays, which run docker compose. Past either, the check is left to bundling, which refuses the same service, as with no folder.
• A refusal exits 78, not 1. With #3661, a readable failure.json leaves the bundle cache alone.
• A folder value that doesn't match the platform's format is refused as subfolder_not_found, though the contract has the format checked on the platform only, because the kit also runs from jobs the platform never checked.
• extension.source.subfolder comes from the folder the bundle recorded. If that differs from the job's, no document is sent.
• With a folder, credential scanning covers the install root. Only the folder, the folders above it and the code it imports can require a value; everything else shipped is listed as optional. JavaScript and TypeScript imports are followed only when written as a path, and a Compose file above the folder counts only when the runtime names it. The browser-tool check reads the same reach, without sibling folders.

TH-8805 is included (merged in from #229)

#229 (TH-8805) was merged into this branch on 4 Nov, so this PR also brings src/fi/alk/harness/framework_detection.py (1,027 lines after the fixes) and its tests to dev. The wiring for it is in hosted_entrypoint.py, scenario_source.py and providers/agent.py. Please review it as part of this PR. #229's own description has the full detail: #229

What it adds. Every build's agent document says which framework the agent uses: extension.framework = {name, version}, plus label for other. The answer comes only from reading files. The customer's code is never imported or run, and no model is asked. The words are the Studio contract's 10, plus other, unknown and none.

How it was verified:

• TH-8805 as merged from feat(harness): report the agent's framework in the agent document #229 (on f9d3452, before the review fixes below):
  • three change reviews, the last clear (High 0, Medium 2, Low 4);
  • 75 planted bugs, each caught by a named test. Plant 10 no longer applies, since the line it changed is gone; P3, P4 and P4b below now cover that rule;
  • focused pytest on 3.11 (7 failed, 652 passed) and 3.12.14 (1 failed, 658 passed). The failures are the base's, by name: 6 need livekit on 3.11, and one is test_grade's cap test;
  • 3.12.3, the sandbox's version, for the detector and the scenario source: 274 passed.
• On this branch's 3e4e3ec, before the fixes: 762 passed on 3.12.14, over its files and the neighbouring ones.
• The three review fixes, on 3e4e3ec plus the fix PR:
  • 10 planted bugs, each caught by its named test;
  • on 3.12.14, the detector, hosted entrypoint and scenario source pass (262, 136 and 23);
  • on 3.12.3, the detector and scenario source give 285 passed;
  • one change review (Lane Q), clear: High 0, Medium 2, Low 4.
• Not seen by a reviewer:
  • feat(harness): report the agent's framework in the agent document #229's round 3 two Medium test additions and its Low fixes;
  • the landing here;
  • after the fix review: two test rows, a test comment, this description's edits, and the repeated tests removed at your request on fix(harness): answer the framework comments on the subfolder PR #248.

Your review's three framework comments are fixed in #248 (#248), into this branch:

• .js, .jsx, .mjs, .cjs, .mts and .cts now count as other source. So a Python folder with them, outside test, example and similar folders, reads unknown unless a framework is named. File names are matched case-sensitively, as before, so INDEX.JS still isn't counted;
• any blocker gives unknown before the chat-model rule;
• a repository-origin process is scanned in the source.

Known limits:

• Nothing reads extension.framework yet (platform #3661).
• unknown is answered whenever the check isn't sure.
• One folder's answer only; finding the agents in a repo is TH-8950.
• A few frameworks outside the lists, dynamic imports and from livekit import * read as none.
• Docker agents report version: null.
• Timings and memory are laptop measurements: about 480 s at worst; memory 858 MB measured, about 1.1 GB estimated with the harness.
• A plain Python agent beside JavaScript outside test and example folders reads unknown.
• A LangChain agent that only uses a chat model (no agent or graph builder) reads unknown when anything blocks. That includes test or eval files that import a framework, its own agents package, and JavaScript or TypeScript outside test and example folders, including config and built files such as tailwind.config.js, static/js/ and web/dist/.
• setup_hash includes extension.framework, so an agent whose answer changes gets a new hash on its next build.

Landing with TH-8806 and TH-8805

#224 is merged into dev (dc25cad), and dev is merged into this branch at c07f4bb, not rebased. This branch uses #224's EXIT_REFUSED and failure_file.py. submit_contract's browser-tool scan reads the agent's folder, what it imports and the files directly above it, not sibling folders, with a missing folder counting as nothing found. TH-8805 (#229) was merged into this branch on 4 Nov; its fixes for your review are in #248, into this branch.

Landing with #274 (TH-9057, TH-9039: Claude's per-turn cost)

#274 changes backends/claude.py to report each turn's cost, and the shape of the price table. Whichever of #274 and this PR lands second updates tests/harness/test_agent_reading.py:

• Removes the subtraction: _ReadingStage._own and the running_totals flag go, and three tests move to per-turn figures from the fake backend: test_no_contract_after_the_reminder_is_incomplete, test_an_error_flag_without_a_status_still_gets_the_reminder and test_claude_running_totals_are_counted_once.
• Fixes the price-table helper: _gemini_priced changes to rate_in, rate_out = current_rates(_PRICED_GEMINI). Two tests use it: test_a_gemini_call_limit_is_incomplete_with_no_reminder and test_a_gemini_call_limit_after_a_malformed_worker_reply_gets_no_reminder.
• Behaviour change: after fix(harness): record each Claude turn once, never the CLI's estimate, and date the price table (TH-9057, TH-9039) #274, a reading writes complete: false only for a model missing from the price table.
• Tripwire: fix(harness): record each Claude turn once, never the CLI's estimate, and date the price table (TH-9057, TH-9039) #274's test_a_claude_reading_adds_per_turn_figures_as_they_come fails while _own is still here after the merge.

Verification

• uv run ruff check .: not run on the whole repo. ruff check on every changed .py file passes.
• Focused pytest target: named files only, 4005 passed, 16 failed, 7 skipped, on Python 3.11. CI's exact list on 3.12.14: 3646 passed, 3 skipped, 2 deselected. All 16 also fail on the base commit and none is about folders (the 11 architecture failures went with test(harness): fix the nine hosted-GitHub tests and guard what their fixes left unchecked #272). The six changed-area files: 1333 passed on 3.12.14 and 3.12.3, 1332 and 1 skipped on 3.11.
• Full uv run pytest -q: not run, because of the laptop limit. CI runs the list in sdk-smoke.yml, which now includes this change's test files except tests/test_harness.py. One file runs with 2 tests deselected, which fail on dev too: TH-8978. The nine in test_harness_architecture.py run again since test(harness): fix the nine hosted-GitHub tests and guard what their fixes left unchecked #272 (TH-9444).
• TypeScript build/test: not touched.
• agent-learn release-proof: not run.
• Planted bugs: 363 now (plants-34). On the reviewed head, 355 of 356 were caught, the one left being the known equivalent below; the 22 for the last commit's lines are all caught. Before them: the original 128, re-run by reviewers through the fix rounds (124 caught on the last run; the rest changed by design or no longer able to change behaviour), and 250 on the later fixes, of which 249 are caught on this code; the one left removes a check whose answer another check already gives.
• Cold reviews: two on the main change, the second clear; then three rounds on the fixes after it, none clear. Khushal asked for a new count after the third: its first round found two more ways a crafted YAML file could stall the save, which he graded High, and its second found two more (a long base-60 number, and plain YAML read in full before it was counted), which he also graded High; its third found one more ([ nested hundreds deep, about 4 s under the save's 128 KB), also High. He asked for a new count after it. All five are fixed here, and so is an alk.yaml read that trusted the file's reported size. The new count's first round found that crafted files still held the save about 1.2 s and that a %TAG prefix could take 1.5 GB, plus an older endless read of a linked .py; so the save's checks now run off the event loop, and both are fixed. Its second round found that Python's parser still held the conversation, 13.5 s on one crafted .py (on base too), that a value from alk.yaml repeated in Compose could take a save past 1.5 GB, and that of two contracts sent together the first could win. All three are fixed, with the smaller findings. Its third round found a crafted .ts file holding the conversation about 20 s and a Compose wait of about 4 s; both are fixed, and Khushal accepted the Python ceiling (1.6 s on 3.12, 4 s on 3.11) and asked for a new count. That count's first round found five more ways crafted files could stall the save: a Unicode fold on non-ASCII files, the parser folding crafted names on 3.11 and 3.12.3, a chain of nested packages, deep build paths checked on every service, and an alias tree in network_mode. All five are fixed here, with its Mediums and Lows; on the reviewers' own files every save took at most 0.4 s. Its second round found two more: one long line of short tokens held the tokenizer 3.8 s on 3.12.3 (on base too), and a link to a deep folder made 4,500 builds take 9.7 s, since a lookup through a link wasn't counted. Both are fixed: each token is charged its line, and resolving charges each lookup on disk it makes. So is the walk's memory at bundling's limits (1.5 GB at 20,000 names, now under 300 MB), with its other Mediums and Lows, and a sandbox test that the first round's off-loop credential walk made hang about one run in six, which now stubs its job. Its third round, the last, found one more: link targets padded with empty parts, which the resolver walked uncharged (5.25 s on 3.12.3). Khushal chose to fix it and allow one more round on that commit: each empty, . or .. part is now charged, and the save takes 0.29 s. Its six smaller findings went to a follow-up ticket, since folded into TH-9445. The fourth round, on that commit and on the merge of feat(harness): refuse browser, computer-use and crawler agents #224's latest head, found the merge faithful and two more: links that grow the path as they unwind (4.07 s), and the browser scan reading each folder above the agent's install records with a fresh 64 MiB budget, a path object per row (5.74 s, 718 MB). Khushal chose to fix both here, with its five smaller findings, and one more round on that commit. A path past twice the system's limit is refused (0.07 s now), and the folders at and above the agent share one 16 MiB record budget, rows kept as text (0.13 s now; 0.64 s 8 folders deep). Coding lines are now honoured by the import walk too, and with lone CR line ends. The fifth round, on that commit, found three more, all from it: a punycode coding line made the import walk's decode cost the square of the file's size (9.4 s), an idna one made the walk raise and every save fail, and matching every prefix of a path against the records added 1.8 s. Khushal chose to fix them and allow one more round on that commit: punycode and undefined aren't read, the walk falls back to UTF-8 when a codec can't replace what it can't read, and a record's row is matched only against the file itself. The import walk still charged a name once however deep it went; that went to a follow-up ticket, TH-9442. The sixth round, on that commit, found two more coding-line stalls, both older than it: idna with valid labels (3.6 s, from the merge of feat(harness): refuse browser, computer-use and crawler agents #224), and decoding with replacement, which runs Python's own error handler for each byte in most codecs (2.1 s). It found that reading the root's records first hadn't fixed the fifth round's Medium, and that TH-9442's shape takes 12.3 s, not 2.4 s. Khushal chose to fix all of it here, TH-9442 and the five smaller findings included, and allow one more round. idna isn't read, and the walk decodes strictly before falling back to UTF-8 (both 0.1-0.25 s now). Once the record budget is spent, a browser package counts as installed, which leads to a second look, not a refusal. A file is checked against the nearest four folders' records. At the save, the walk stops at 100,000 lookups on disk, a file found charged for each part of its path; honest folders use under 9,000. On TH-9442's 12.3 s shape the walk takes 1.4 s, and the save 4.9 s. The rest is the folder's 31,000 files, which a whole-repository save on feat(harness): refuse browser, computer-use and crawler agents #224 takes as long over. Hawk also found, reviewing a later plan, that the save resolved paths the contract names without charging the lookups: a crafted prompt path took 2.1 s, any number of them could be named, and the runtime's paths and tool modules were resolved the same way. Khushal chose to fix it here as a second commit, which joins the seventh round: one budget of 20,000 lookups covers every such path in a save (honest contracts use under 900), past it the save says so, and a path no system can open is answered rather than raised. Dileep's review of 10 Nov is answered in a third commit, which also joins the seventh round: credentials for a uv workspace member, instrumentation that follows the folder's imports, bind mounts and env files checked like builds, a folder that links to its own parent refused, a narrower catch with a logger in the sibling check, and the import walk saying when a limit stopped it. Replacing the per-scan budgets with one subprocess and a deadline is TH-9445. The seventh round (Owl) found no regression in either commit, and three more crafted stalls: imports through committed folder links escaping the walk's lookup budget (11.6 s), unicode-escape files (7 s), and a multi-megabyte path in the save's resolver (7.4 s). All three are fixed in a fifth commit: at the save an import through a folder link isn't followed, unicode-escape isn't decoded, and a path past twice the system's limit is refused first. Its smaller items and the many-files cost are folded into TH-9445. PR 224's last heads are merged in, and Dileep's new point on it, that a file left undecoded could hide a browser import, is answered the same way here. Under Khushal's 10 Nov rule the developer runs the cold reviews: three fresh reviewers spot-checked the fourth and fifth commits and the merges of feat(harness): refuse browser, computer-use and crawler agents #224. They found a deep script import that crashed the save (the link check now runs after the depth cut), a merge stall of 9.2 s (the raw searches now run under a 1 MiB total), an import spelled in unicode-escape escapes (decoded as Python does under a 4 MiB total, the rule both PRs now share), and a non-ASCII coding line that hid a file. One more fresh reviewer checked those fixes and found an 18.6 s stall the coding-line fix had added, now 1.9 s, and a CRLF case; both are fixed. The rest is ticketed: TH-9448 (the cost of files left undecoded) and TH-9490 (three rarer codecs).

Dileep's third review (10 Nov, four comments) is answered here, with six self-run cold review rounds of two fresh reviewers each, under Khushal's plain Lane N rule. The first commit makes a workspace member the agent depends on part of the walk (Khushal: "fix it now", in place of the known limit), follows committed folder links at the save, charged, builds instrumentation's reach from the whole walk, and guards the 3.13 assertion. Rounds 1 to 4 each found crafted save stalls in those lines, from 1.8 s to 53 s: pyproject files parsed uncharged, member globs through links, every name tried against every member, link targets walked uncharged, a coding line's cookie pattern over a 128 KB line, eight escape lookbehinds at every byte, a long uv path source, the sources table rebuilt per requirement, a member re-read for each new extra, and the folder walk searching back down to a missing folder for each candidate. Rounds 1 and 2 also found that every member, a sibling agent's included, counted as the agent's code; the walk now follows only what the build installs, compared with uv itself. Each was fixed and re-measured, at 0.06 to 1.4 s on 3.12.3. After round 4, Khushal set the line for this PR at 5 s: a crafted stall these lines bound but don't remove is a TH-9445 item below it. Round 5 found two above it (a cookie search re-reading a line, 8.5 s, and the extras regex costing the square of a requirement, 51 s), both fixed. A design review of TH-9490 on this branch (Pine) found that the cookie cap refused a file whose real cookie came first, and that plain files ahead of the agent's imports could spend the parse total and hide a shared import with no log; both are fixed. Round 6 cleared, with nothing above Low; its six Lows (tests and two small simplifications) are fixed in the last commit, which had no round, as the rule allows. The merge of #272 (TH-9444, merged into this branch at c1685ec) brings only tests/test_harness_architecture.py, and sdk-smoke.yml runs its nine tests again. Tickets from these rounds: TH-9509 (a stopped walk records instrumentation none), TH-9513 (--all-packages against the walk), and the folder count, now part of TH-9445.

Dileep's review of 11 Nov is answered in 1132192, with two rounds of one fresh reviewer each, under the leaner rule for a small change. The first round wasn't clear; the second cleared, and its two code Lows are fixed with no further round.

• The comments this branch added, in src and tests, no longer name reviewers, rounds or people.
• A uv workspace member past the 256 read for names is read when a requirement names its folder, at most 32 such reads a walk, and the walk still reports cut.
• His two comments on fix(harness): the save's import walk says when it leaves code unread, and reads more of it (TH-9543) #283, which was already merged here, are answered here too:
  • A folder link back into the agent's folder doesn't count as code left unread, unless it leads into a skipped part of the folder.
  • Past 32 folder links the walk says stopped without resolving more. This also bounds an older uncharged resolve: 2,000 linked chains took 19.6 s before, and take 0.49 s now.
  • A dangling .py link still counts as unread: checking it would follow its chain uncharged, and some targets raise.
  • Any .py holding import that doesn't parse counts as unread, whatever the reason.

Merges with dev:

• 623d7de, resolved by hand (mechanical): src/fi/alk/harness/sandbox_server.py imports both dev's SecretResolutionError and this branch's folder helpers, and tests/test_harness_sandbox_server.py keeps both sides' new tests.
• 4385a5f dropped that import, and f8a960d restores it.
• cde91b3 joins the two merges: the same import line, and the TH-9491 test (from 71776fa), kept once.
• test_public_github_branch_url_populates_job_ref and test_private_github_submission_becomes_hosted_job fail on dev alone (github_commit_sha_required, job.py:413). CI deselects them (TH-8978).

CI's list, run on a Mac with CI's install (3.12.14), at cde91b3: 4738 passed, 3 skipped, 2 deselected, 1 xfailed. The changed-area files on 3.12.3, on a Mac: 2243 passed.

• Skipped: test_certify_only.py's real seed test (3 cases). It needs ALK_TEST_POSTGRES_DSN.
• Expected failure: test_hosted_github_route_accepts_a_submitted_value. Submit doesn't yet give hosted jobs the dedicated_vm runtime.

Since then:

• The branch: fix(harness): keep a block comment open across lines, and save no prompt from unconfirmed lines #289, [TH-9448] fix: no traceback at any log level when the browser scan skips a file #291, [TH-9491] docs: answer Dileep's review of #278 #292 and [TH-9449] fix(harness): answer Dileep's P3 comments on #273, #275, #277 and #285 #293 landed on this branch, and dev is merged in again.
  • In ababd8b, resolved by hand and mechanical: sdk-smoke.yml keeps this branch's list, which already holds dev's three sandbox-server lines, and tests/test_harness_sandbox_server.py keeps both sides' new tests.
  • 4ec0bfc merges the branch's new head on top.
• c0e51b7, a test-only fix:
  • On CI, test_a_crafted_python_file_does_not_hold_up_the_conversation failed at 0.38-0.40 s against its 0.25 s line.
  • A diagnostic run showed why: a full garbage collection of the test process's own heap, about 706,000 objects left by the tests before it, landed inside the save's parse and took 0.3 s. The save's own hold was 0.05 s, and Linux runs of both commits matched.
  • The three loop-stall tests now collect, then freeze that heap while they measure. The save's own collections still count.
• Tests: CI's list at 4ec0bfc, 4828 passed, run on a Mac with CI's install. The two changed test files at c0e51b7, 915 passed.

Fix-only checks after the last clear reviews

Each commit made after a review round had cleared got a fix-only check by one fresh reviewer, and each was CLEAR with no findings:

• cf3ebb4, round 6's six Lows: each fix's test fails on its plant.

• The part of 1132192 after its second fix-review round cleared:

  • case-folded skipped parts;
  • slicing the path parts;
  • reworded test headers.

• f8a960d, the restored SecretResolutionError import. Nothing else that merge touched is missing.

• The merges 623d7de, cde91b3, ababd8b and 4ec0bfc. Each was recomputed and compared. Every hand resolution keeps both sides, and no test, CI-list entry, import or code from either side is lost.

• c0e51b7, the GC freeze in the loop-stall tests.

• This commit's sdk-smoke additions. CI now runs every test file this PR touches:

  • added here: tests/test_harness.py, test_contract_model.py and test_contract_writer.py;
  • the two provider test files were already run, through tests/providers.

  The first check wasn't clear on one point: test_contract_model.py's timing checks had no guard against a garbage collection of the earlier tests' heap. Its fix freezes that heap for each test in the file, and a second fix-only check covered it. The two checks nearest their 0.5 s line, at about 0.27 s, are:

  • test_a_crafted_file_of_block_comments_does_not_hold_up_the_check[a-file-of-openers];
  • test_a_quote_that_grows_when_normalised_costs_little. It runs under tracemalloc, and once read 0.477 s on a loaded machine. If it fails on CI, it is deselected by name.

  Six tests/test_harness.py tests are deselected, because they fail on dev (70fc43b) the same way. They're named in the workflow:

  • test_hosted_claude_session_uses_durable_resume_and_restricted_tools;
  • test_a_skill_only_names_tools_its_stage_actually_has;
  • test_connect_only_provider_world_cannot_create_shadow_state;
  • test_the_system_prompt_stays_within_budget;
  • test_a_save_refused_for_the_suites_shape_is_accepted_on_the_second_try;
  • test_dropping_every_scenario_needs_an_explicit_everything.

• The deep-flow test reads no clock now. test_deep_flow_nesting_does_not_hold_up_the_save asserted the save took under 1 s, and failed once on a slow runner (test(harness): a Deep Agents reference agent for the hosted path (TH-8810) #306's run 43166681740). It now counts the YAML loader's token fetches, under 100, and checks that the refusal comes at the sixteenth open group.

  • With the limit, the save takes 19 fetches. Without it, a sixth of the file takes about 19,000.
  • The fix-only check (CLEAR) also showed it fails on a limit moved to 64 or 400, and on a refusal held back past 1,000 tokens. The old timing let all three through.

Checklist

• Public API names stay under agent_learning.*, agent-learn, or @future-agi/agent-learning-kit
• New user-facing workflow has docs or an example: the platform field isn't on yet
• New release claim has executable gate coverage
• No hosted-service dependency was added to release-gated examples
• Security-sensitive output is redacted or justified

Refs: TH-8804

[cdileep23]

Please check the comments.

[cdileep23]

A subfolder on a non-GitHub source crashes instead of refusing (P3).

subfolder_requires_github_source fails job parsing, so the hosted run exits 2 (crashed) with no terminal failure, and authoring main exits 1 with no failure.json. The platform sees a crash, not a clear reason.

Related dead branch: scenario_source.py echoes the folder for ARCHIVE sources too, which can never carry one.

Fix: report it as subfolder_not_found (or its own code) through the failure file and terminal event, and drop ARCHIVE from the echo branch.

[khushalsonawat]

Not changed. #3661's serializer refuses this with a 400 first, so only a hand-written job reaches the parse, and a bad job file already exits as crashed by design. The ARCHIVE branch can't be dropped: it builds the whole document for uploads, not just the echo. Leaving this open for you.

[cdileep23]

A failed document build reaches the platform as subfolder_ignored (P3).

Any exception from build_authored_agent is only logged and registration continues with no document. On a subfolder build the platform then fails the job with subfolder_ignored, which hides the real cause (for example an AgentBuildError).

Fix: when job.source.subfolder is set, re-raise (or fail with the real code) instead of continuing without the document.

[khushalsonawat]

Not changed. At #3661 225536bf0, _check_subfolder_echo returns early when no document was sent, so a failed build can't surface as subfolder_ignored. Failing a folder build whenever the document can't be built would be a new behaviour. Leaving this open for you.

[cdileep23]

On the run path the folder failure loses retryable and details (P3).

_fail builds the terminal failure from domain, stage, code and message only, so details.subfolder (and retryable: false) never reach the platform for a run that reports through the terminal event on exit 0. It isn't retried either way, so this is about what the user and logs see.

Fix: let _fail pass retryable and details through, and send them here from unusable.

[khushalsonawat]

Not changed. HarnessFailureSerializer at #3661 has only domain, stage, code and message, and only the failure-file path adds retryable and details, so as I read it these keys wouldn't reach the platform from the terminal event. It isn't retried either way. Leaving this open for you.

[cdileep23]

Please check the comments.

[cdileep23]

Please check the comments.

[cdileep23]

Please check the comments.

[cdileep23]

Keys read by a src/-layout uv workspace package become optional (P3).

python_bases is only the folder, folder/src, the root and root/src, so workspace members are never resolved.

Checked: root pyproject.toml with [tool.uv.workspace] members = ["agents/a", "packages/shared"], agents/a/agent.py doing from shared.store import db, and packages/shared/src/shared/store.py reading SHARED_DB_PASSWORD. A root build marks it required; agent_subfolder="agents/a" marks it optional. Known limits only mentions this for JS/TS, but it's a common Python monorepo layout.

Fix: add each workspace member's root and src/ from [tool.uv.workspace].members to python_bases, or list the limit.

[khushalsonawat]

Kept as a known limit, by Khushal's call. A package that only a uv workspace (or a path alias) can resolve isn't followed, so the keys it reads stay optional, and a missing one fails at the agent's first call. It's listed in the description under Known limits, "Imports the walk can't see".

[cdileep23]

Please check the comments.

[cdileep23]

P3. Following up on the shared JS imports thread: relative files and index.* work now, but a package folder is still followed only through index.*. Its package.json main or exports is ignored.

Failure: agents/a/index.js does require('../../packages/shared'), and packages/shared/package.json has "main": "lib/llm.js". A key read in lib/llm.js is required on a whole-repo build and optional on a folder build, so the agent dies on its first call. The browser check also misses puppeteer required there. Known limits only cover names that need a workspace or path alias; this one is written as a path.

Fix: when the target is a folder, try its package.json main (and a string exports) before index.*, with tests for both and for the browser check.

[cdileep23]

P3. Freezing the heap helps, but the four took < 0.25 asserts (:2677, :2809, :2951, :3184) still race the runner, and CI now runs this file. Under load one of them failed 4 runs in 5 at 0.29 s. Use QUICK_SECONDS for those too.

[cdileep23]

P2. The description covers the subfolder build and framework detection, but not the reading and scan work merged in through #245, #268, #269, #271 and #288: this module, agent_scan, scan_walk, scan_ai, scan_document, reading_entry, seed_contract, model_citation and the find-agents skill, about 8k lines of source and 15k of tests. None of those modules or their tickets appear in the body, and most of it reads as a log of review rounds. Please update it to list what the PR now ships, so whoever merges it into dev knows what's going in.