Skip to content

[TH-8951] fix: log a refused URL once per document, and pin the reading's stop wiring - #271

Merged
khushalsonawat merged 1 commit into
feat/TH-8804-source-subfolderfrom
feat/TH-8951-d1f968cc-fixes
Nov 11, 2026
Merged

khushalsonawat merged 1 commit into
feat/TH-8804-source-subfolderfrom
feat/TH-8951-d1f968cc-fixes

Conversation

@khushalsonawat

Copy link
Copy Markdown
Member

Description

Follow-up to #245, whose last commit d1f968cc (answering Dileep's 5 P3 comments) went out unreviewed. A cold round on d1f968cc alone has now run, with two fresh reviewers. It cleared with nothing above Medium. This PR carries the fixes from that round that apply in the sandbox.

Type of Change

  • Bug fix

Changes

  • providers/agent.py: _redact now logs a refused URL once per call, not once per string.
    • A crafted contract at the 5 MiB cap held 582,527 refused strings. That produced 25 MB of log, and the build took 4.77 s.
    • It now logs the same class-only line once: 1 record, and 1.10 s.
  • test_agent_reading.py: the stop-budget test now also checks that READ_AGENT is wired to the 10 / 60 / 20 s bounds, not only that the constants sum to 90.
  • Where the scan_walk.py figure comes from: you asked in review for the source of the figure to move out of the comment, so it is now in the commit message. The figure is about 17.5 bytes per edge entry, so the 5,000,000-edge cap is about 90 MB, from Dileep's measurement on 10 Nov.

Dropped from the round: its findings about Ctrl-C and KeyboardInterrupt. The reading runs only in a sandbox, where no interrupt is sent (owner's ruling).

Found, not fixed here: a SIGTERM to the reading kills it at once with exit -15, because the reading path installs no handler.

  • After the save, contract.json stays but agent.json is never built.
  • Before the save, no failure.json is written.

That needs the platform's answers first: whether such a stop is retryable, and how long it waits before SIGKILL. It is being raised separately.

Testing

  • tests/providers/test_providers_agent_build.py 11, tests/harness/test_scenario_source_provision_agent.py 24, tests/providers 52, tests/harness/test_agent_reading.py 105 and tests/harness/test_reading_entry.py 61: all passed.
  • Plants, each restored by sha256:
    • logging once per string again fails test_many_refused_urls_in_one_document_log_one_line;
    • READ_AGENT = _read_agent(interrupt_wait=200.0) fails the stop-budget test.
  • ruff is clean.

Related Issues

Refs: TH-8951

…ing's stop wiring

From the cold round on d1f968c (cleared, nothing above Medium):

- _redact logged one warning per refused string. A crafted contract at
  the 5 MiB cap held 582,527 of them: 25 MB of log, and the build took
  4.77 s. It now logs the same class-only line once per call (1 record,
  1.10 s).
- The stop-budget test re-stated the three bounds. It now also checks
  that READ_AGENT is wired to them, so a reading built with another
  interrupt wait fails it.

Moved from a code comment in d1f968c, as asked in review: the
scan_walk.py figure of about 17.5 bytes per edge entry (so the 5,000,000
edge cap is ~90 MB) is Dileep's measurement, 10 Nov.

Refs: TH-8951
Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
@khushalsonawat
khushalsonawat merged commit e7cc68b into feat/TH-8804-source-subfolder Nov 11, 2026
9 checks passed
@khushalsonawat
khushalsonawat deleted the feat/TH-8951-d1f968cc-fixes branch November 11, 2026 05:40
khushalsonawat added a commit that referenced this pull request Nov 11, 2026
Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
khushalsonawat added a commit that referenced this pull request Nov 11, 2026
Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants