Skip to content

Hand any blocked agent to Captain during explore and freesail - #285

Merged
DavertMik merged 8 commits into
mainfrom
blocked-error
Oct 31, 2026
Merged

DavertMik merged 8 commits into
mainfrom
blocked-error

Conversation

@DavertMik

@DavertMik DavertMik commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Generalises the planner-to-Captain handover from #282 and #283. When an agent reports it can't go on, Captain gets the full situation and decides how to get past it, then the work continues.

Changes

  • BlockedError (src/utils/error-page.ts) is the base error for "an agent can't go on". It carries two texts:

    • message: a short line for the logs.
    • request: what Captain reads. It covers what the agent was asked to do, where it stopped and why, what it already tried, and what happens next.

    There are three kinds:

    • PlanningBlockedError: the planner can't plan the page.
    • NavigationBlockedError: the Navigator can't reach a page. Its reason is the Navigator's single lastFailureReason, and the request lists the last 5 commands it tried.
    • TestStartBlockedError: the judge says the open page isn't where the test can start. The check runs only when ai.decisionModel is set, and only a confident "no" stops the test.
  • Captain.resolveBlocker(error, resolved) gives the request to Captain, then runs resolved to check the outcome and returns whether the blocker is gone. A blocker Captain failed on isn't handed over again. A blocker raised while Captain is already working is ignored, so Captain's own navigate and plan() calls never start a second Captain run.

  • Who asks Captain:

    • ExplorBot.visit() asks when the Navigator is blocked. The check is whether the browser is now on the target page.
    • ExplorBot.plan() asks when the Planner is blocked. The check is whether new tests appeared.
    • Tester never calls Captain. When the judge rejects the start page, Tester throws TestStartBlockedError before the test starts. explore hands it to Captain and runs the test again if Captain resolves it, or skips the test if not. The test command skips it without Captain.
    • User interrupts during a running test reach Captain through a supervisor callback that ExplorBot injects into Tester (setSupervisor), in place of setCaptain.
    • Apart from the test start, commands don't wire the handover themselves. explore only skips a page that stays blocked.

The handover now applies wherever Explorbot opens a page, plans a page or starts a test, including explore, freesail, test and TUI /navigate. In interactive runs the Navigator still asks the user before it gives up, so Captain is asked after the user.

Testing

  • Unit and integration tests pass, apart from the 8 failures already on main. New tests/unit/captain-blocked.test.ts covers:
    • resolveBlocker: a resolved handover, not handing over again, and the guards.
    • ExplorBot.visit() and plan(): each with the blocker resolved and not resolved.
  • Ran local bunosh regression:gated with real AI on the first commit of this PR. It passed on attempt 2 of 3: 8 tests planned across 4 features, and 5 of 5 run tests passed. In both attempts Captain took over the blocked planning, signed in and re-planned. Attempt 1 failed only the scenario-coverage check: the planner covered 2 of the 3 required features. The latest commit moves that same handover into ExplorBot.plan(). It has not been re-run against real AI.

🤖 Generated with Claude Code

BlockedError is the base for agents reporting they cannot go on. It
carries a short message for the logs and a request for Captain: what the
agent was asked, where it stopped, why, and what was already tried.

- PlanningBlockedError now extends it.
- Navigator throws NavigationBlockedError when it cannot reach a page.
- Tester asks the judge whether the open page is where the test can
  start, and throws TestStartBlockedError when it is not.

Captain.resolveBlocker() hands the request to Captain and checks the
outcome. It remembers blockers it could not resolve so they are not
handed over again. Explore and freesail route through it; the test
command skips a blocked test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
visit() and plan() on ExplorBot ask Captain when the Navigator or Planner
reports a BlockedError, so commands no longer wire the handover. Tester
asks the Captain it already holds when the judge says the open page is
not where the test starts. Captain ignores blockers raised while it is
already working.

Navigator keeps a single failure reason for both the log message and the
Captain request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tester no longer holds Captain. A start page the judge rejects throws
TestStartBlockedError before the test starts; explore hands it to Captain
and runs the test again, or skips it. The test command skips it.

User interrupts during a running test reach Captain through a supervisor
callback that ExplorBot injects into Tester.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After signing in, Captain called plan() with the blocker text ("Signing
in with valid credentials...") as the focus, so the page was re-planned
around the sign-in step. While Captain resolves a PlanningBlockedError,
the plan tool uses that error's feature and style instead of the ones the
model passes. Outside a blocked re-plan the tool plans what Captain asks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Explorbot Self-Regression

Commit f3db7fa · run

Scenario Result Attempts Time Previous median Change
basic (native) PASS 1/3 5m 49s 6m 23s -9%
experience: control OK — failed as expected 1/1 28s 16s +75% slower
experience: seeded PASS 1/3 44s 42s +5%
slow (native) PASS 1/3 1m 49s 1m 23s +31% slower
gated (native) PASS 1/3 2m 23s 2m 23s 0%

Time is the passing attempt (the last attempt if none passed). Change compares it with the median passing time of the last 1 runs; ±20% or more is flagged.

Attempt details

  • basic (native) attempt 1 — PASS (5m 49s): login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=6, wellFormed=true, keywords=7/3); scenarios: PASS (tests=7/5, features=4/3); tests passed: PASS (5 passed, 0 failed (reporter: 5 passed, 0 failed))
  • experience: control attempt 1 — PASS (28s): control: OK — failed as expected (0 passed, 0 failed)
  • experience: seeded attempt 1 — PASS (44s): seeded: PASS (1 passed, 0 failed)
  • slow (native) attempt 1 — PASS (1m 49s): login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=3, wellFormed=true, keywords=7/3); scenarios: PASS (tests=6/5, features=3/3); tests passed: PASS (1 passed, 0 failed (reporter: 1 passed, 0 failed))
  • gated (native) attempt 1 — PASS (2m 23s): login evidence: PASS (post-login plan=true, post-login research=true); research: PASS (files=4, wellFormed=true, keywords=7/3); scenarios: PASS (tests=8/5, features=4/3); tests passed: PASS (1 passed, 0 failed (reporter: 1 passed, 0 failed))

Session analysis — basic (native):

Session Analysis

Issue search by text works on the issues list: a matching term narrows the table and the URL carries the query. Creating an issue and empty-title rejection were not cleanly confirmed, because their form steps failed in automation, and the status filter and empty-search state passed only after failed steps.

Coverage

  • Pages: /issues, /issues?new=1, /issues/6
  • Features: issue creation, empty-title validation, text search, empty search state, status filter, issue detail navigation

What works

  • Issue search — ET-3 Search for a term matching an existing issue and verify only matching issues are shown

Execution Issues

  • ET-1 Create a new issue from the toolbar and verify it appears in the issues list — Title fill step failed; creation proven only by later checks.
  • ET-2 Attempt to create an issue without a title and verify it is rejected — Submit step failed; validation feedback never checked in log.
  • ET-4 Search for a term with no matches and verify the empty result state — First click step failed; search applied via Enter.
  • ET-5 Filter issues by status and verify the list shows only issues with that status — First status select failed; retry succeeded.

Session analysis — slow (native):

Session Analysis

Creating an issue from the toolbar saves it, and the new row shows the entered title, status, and priority in the issues list. Coverage is limited to this single creation flow.

Coverage

  • Pages: /issues, /issues?new=1
  • Features: toolbar New Issue button, New Issue overlay form (title, status, priority), Create Issue submission, issues list row display

What works

Defects

UX issues

Execution Issues

  • ET-1 Create a new issue from the toolbar and verify it appears in the issues list - passed, but a fill step logged FAILED then was repeated

Session analysis — gated (native):

Session Analysis

Issue creation from the Issues toolbar appears to save and show the new issue as a link in the table. The post-submit checks are weak, so the form-close and table results are not fully confirmed.

Coverage

  • Pages: /workspace, /issues, /issues?new=1
  • Features: New Issue drawer, title and priority fields, issue creation, Issues table

Execution Issues

  • ET-1 Create a new issue from the New Issue button and verify it appears in the issues table - passed despite empty-argument seeInField and contradictory unconfirmed-table note.

DavertMik and others added 2 commits October 31, 2026 21:01
DavertMik and others added 2 commits October 31, 2026 21:12
@DavertMik
DavertMik merged commit dd6d93e into main Oct 31, 2026
2 of 3 checks passed
@DavertMik
DavertMik deleted the blocked-error branch October 31, 2026 19:19

This branch is waiting to be deployed

1 waiting deployment
regression — 81e7e5c9 Waiting Oct 31, 2026 by DavertMik via regression #199
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant