[khushalsonawat]

Description

These are follow-ups from the last review of #224, the check that refuses browser and computer-use agents (TH-8806), all in agent_type_check.py.

The one with real weight is a log leak, fixed here.

• The cause: a .py file with a NUL byte after an indented block makes the Python 3.12 and 3.13 tokenizer raise SystemError. The scan's skip handler logged that at WARNING with its full traceback. The traceback quoted the customer's source line holding the NUL.
• The reach: this got into production logs. The hosted root logger is at INFO, and --read-agent-only falls back to Python's last-resort handler at WARNING.
• The scale: 20,000 such files gave about 2,850 tracebacks (5 MB) in one scan.
• The fix:
  • the tokenizer's SystemError is caught with its other errors, at DEBUG;
  • the skip handler logs the error type only, once per scan, plus a count line;
  • the path and the trace go to DEBUG.

Type of Change

• Bug fix

Changes

• Item 3, the log leak (above). The only WARNING line left in this file names the job's own subfolder setting (_walked), never a file or a line. credentials.py logs the same at WARNING.
• Item 1. A folder holding exactly the same package records as one already read shares that folder's rows. They are unread and never copied, for an eighth of each record's size (at least 4 KiB each).
  • 21 copies of a 160-package environment now fit in the 16 MiB record budget, against 9 before.
  • A near copy, with one package more or fewer, is read as before.
  • The agent folder's own walk in the subfolder scan shares too; the records at and above the agent don't.
• Item 5. The scan's cache key sums every read file's modified and change times, with the count and size. Neither a restored mtime nor a file dated in the future can hide an edit.
• Items 4 and 10, tests only:
  • a nested folder's records don't take the top's own modules;
  • both 16 MiB record budgets and the 32 MiB Python total are pinned at their real values;
  • a refused-codec file is searched whole, for an import, a marker and the toolkit;
  • a folder holding only a folder inside a test-named folder is read.
• Ruled out, each pinned by existing tests:
  • Item 2 was fixed by 7726ded.
  • Item 6 is intended. Its turn-limit bound (70 turns, or 140 for a generic job read unattended) is now in HOW-IT-WORKS.md; the ticket's "bounded by the first save" was wrong.
  • Item 8 is bounded by the 32 MiB Python total, at about 0.75 s against the 3.6 s measured then.
  • Item 9 is whole names within the raw-search total, by design past it.
  • Item 7 was closed by feat(harness): build an agent from a repo's subfolder #225 and pinned by fix(harness): decode a Python file before searching it when its codec can spell a name (TH-9490) #276.
• The cited review reports (dileepfix3-self-r1/r2) are missing. The items were rebuilt from the code, the TH-8806 ledger and fresh probes.

Known limits (Lows, kept here and not as tickets)

• Sharing. Two folders whose records match by name and size but differ inside share rows. The worst case is one model question lost or added, never a refusal, and a crafted repo can already hide any file it names.
• Memory. Sibling folders with different records keep their rows for the whole walk, which stays within the 16 MiB rows bound: 16 different 1 MiB siblings peaked at 162 MB, against 24 MB before. Exact copies use less than before.
• Overdraw. A shared folder whose charge overdraws the budget returns no rows, where the old code read part of it. That means more files are scanned, at worst one extra question.
• One defence no test can reach. Pass 2's whole = False on the budget break can't change an outcome, because no later folder can match once the budget is spent.
• The cache key on Windows. The modified-time half of the key matters only where the change time is the creation time. CI has no Windows job.
• Catches the tests don't pin:
  • how broad the skip handler's except Exception is;
  • the older ValueError and SyntaxError in the tokenizer catch.
    Narrowing either adds at most a type-only WARNING, never a leak. P3a (dropping SystemError) is caught on 3.12 and 3.13 only, because 3.14 raises TokenError there.
• SyntaxWarnings. The shared reader's SyntaxWarnings still print to stderr, naming <unknown>, with no path or line. That is the shared reader's, in fix(harness): decode a Python file before searching it when its codec can spell a name (TH-9490) #276 and fix(harness): give a browser scan its limits cut a second look (TH-9512) #281.

Testing

test file	passed
test_agent_type_refusal	678 (651 + 27 new)
test_contract_tools_subfolder	180
test_executor_completed_counts	8
test_agent_reading	105
test_builder_chat	60
test_hosted_authoring_entrypoint	12
test_harness_architecture	55

Each file ran on its own, and none failed.

Planted bugs: 35 one-token or one-line breaks, each run in a scratch copy with stale bytecode cleared, each turned its named test red. The reviewers re-ran them on the final code and added their own.

Stability: the timing and memory rows passed 10 of 10 repeat runs.

Review:

• a design review, two rounds (the first was NOT CLEAR, on a test that was flaky on Linux);
• a plan review, one round;
• one change review round;
  each by two fresh reviewers. All the final rounds were CLEAR, with nothing above Low in the change review.

Overlap with #281 (TH-9512)

• What conflicts: item 3's except block and test_a_file_that_raises_something_unexpected_costs_only_that_file clash with fix(harness): give a browser scan its limits cut a second look (TH-9512) #281. Whichever lands second rebases. Once both are in:
  • fix(harness): give a browser scan its limits cut a second look (TH-9512) #281's cut by skipped N line replaces this PR's count line;
  • that test, and the new skip test, expect fix(harness): give a browser scan its limits cut a second look (TH-9512) #281's cut line.
• What doesn't: items 1, 4, 5 and 10 merge cleanly onto fix(harness): give a browser scan its limits cut a second look (TH-9512) #281's head. fix(harness): decode a Python file before searching it when its codec can spell a name (TH-9490) #276, fix(harness): bound the scan walk by what it holds, merge its two readers, test its untested branches (TH-9449, TH-9450, TH-9451) #273, fix(harness): bound the framework check's walk by the folders it lists (TH-9511) #275 and fix(harness): bound the scan's first look for a file (TH-9525) #277 don't touch agent_type_check.py.

Related Issues

Refs: TH-9448

[cdileep23]

Please check the comments.