Skip to content

Require explicit work orders before auto-starting tasks - #1845

Merged
AbirAbbas merged 13 commits into
Agent-Field:devfrom
mikemikimike:fix/1804-auto-spawn-intent
Oct 20, 2026
Merged

AbirAbbas merged 13 commits into
Agent-Field:devfrom
mikemikimike:fix/1804-auto-spawn-intent

Conversation

@mikemikimike

@mikemikimike mikemikimike commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Require clear authorization to do work now in both route judgments before automatically starting a task. Polite requests phrased as questions ("Could you fix the failing tests?") still count; feasibility questions, explanation requests, ambiguous questions and future direction stay in chat.
  • Prevent a stopped task's result wake from starting the same request again, while keeping a new human message eligible for normal judgment.
  • Clarify task routing behavior in the chat manual and add regression coverage for both cases.

Validation

  • go build ./... passed.
  • gofmt -d internal/session/route_judge.go internal/session/route_judge_test.go produced no diff.
  • go vet over the Windows production sources in internal/session passed.
  • go generate ./internal/manual passed.
  • go test -count=1 ./cmd/codeaf-changes passed.
  • go test -count=1 -p 2 -timeout=15m ./... fails on existing Windows-incompatible test sources and platform assumptions; the same command also fails on clean dev.
  • The ordinary internal/session test package cannot build on Windows because existing tests depend on Unix-only packages and syscalls.

Fixes #1804

#1804

@ZeroPoint95 ZeroPoint95 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mikemikimike, the unwanted starts and stopped-task respawn reported in #1804 are worth fixing, and the stopped-result guard makes sense. I’m requesting a change to the routing rule before merging.

The shared prompt currently says: Every question is false, including "can we..." and "could you...". That treats being a question and being a request to act as mutually exclusive, even though both can be true:

  • “Could you fix the failing tests?” is both a question and a request to act
  • “Can you implement headless screenshots and add tests?” is likewise both a question and a clear request to act
  • “How would headless screenshots work?” asks for an explanation
  • “Can we get screenshots without a display?” is ambiguous without context and should not automatically start work
  • “We should support screenshots eventually” expresses future direction, not authorization to begin

Please distinguish requests to perform work now from feasibility questions, discussion and brainstorming, rather than requiring a non-question grammatical form. Clear polite requests should remain eligible for normal task routing; ambiguous messages should stay in chat for clarification. Keep the stopped-result guard so stopping a task cannot itself restart the same request.

Please update both routing prompts and the manual consistently. The current authorization test checks that the rule’s text appears in the prompts, but doesn’t test how the classifier handles these examples. Add positive and negative examples covering the distinctions above, and provide a before/after classifier evaluation with the model and settings recorded. Preserve the regression showing that a stopped-result wake cannot respawn the task while a fresh human request remains eligible.

The goal is to prevent unauthorized work without suppressing ordinary ways users ask us to do things.

mikemikimike and others added 4 commits October 19, 2026 16:56
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Separate feasibility questions from explanation requests in the shared route
contract and the searchable chat manual. Pin examples with their complete
classification clauses, and prove stopped-result suppression does not block
fresh requests or normal remediation.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Give the stopped-result rule its own section so the manual context includes
both the restart guard and the fresh-human-request exception. Pin the actual
retrieved answer, since merely finding the long tasks page hid the rule beyond
the section limit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AbirAbbas

AbirAbbas commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

@mikemikimike thanks for the fix and the follow-up commit. Since this had been waiting on another round, I've taken it over and pushed on top of your branch (no history rewritten).

@ZeroPoint95, here's where each of your points landed:

Requests to act vs. questions. 41a2e41 had already dropped the blanket "every question is false" rule. On top of that, feasibility ("Is headless screenshot capture possible on CI?") and explanation ("How would headless screenshots work?") are now separate examples in the shared contract, which both route prompts use, and in the manual. Authorization has its own heading in the manual, so search reaches it. The prompt test now ties each example to its full classification clause, so a stray "does not" elsewhere in the prompt can't satisfy it.

Stopped-result guard. The production guard is unchanged. The regression now covers a same-request stopped wake (no classifier call, no new task), surrounding whitespace, a fresh human message with the same words (still eligible), an unrelated stopped task, and ordinary non-stopped remediation. With route_judge.go and tasks.md swapped back to dev, both stopped cases fail with start = true, want false. While checking this, I found the manual's stopped-result rule was being cut off from the context the chat actually retrieves. It now has its own heading and a test that checks the retrieved text.

Before/after classifier eval. This used the real prompt constants from dev (26662f1) and this head, gpt-5.6-luna, reasoning effort low, no tools, a fresh context per call, the review's five examples plus ten more (act / don't-act / ambiguous), and 3 trials per prompt per revision.

dev this PR
Review's 5 examples, both prompts 26/27 completed correct (the pre-turn read said no once to "Could you fix the failing tests?") 30/30
All 15 examples 72/84 completed correct 85/86 completed correct

There were 10 transport timeouts across the 180 calls. They're excluded and not counted as correct. The one remaining miss is the pre-turn read of "Could we clean up the configuration loader across all packages?" (ambiguous), which the post-turn read rejects all three times. The settings and raw results can be shared if you want to rerun it.

make pr-ready passes on Go 1.26.5, and current dev is merged in.

@AbirAbbas
AbirAbbas requested a review from ZeroPoint95 October 20, 2026 13:52
AbirAbbas
AbirAbbas previously approved these changes Oct 20, 2026

@AbirAbbas AbirAbbas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at 01b4701: both route prompts and the manual separate requests to act now from feasibility/explanation/ambiguous/future-direction messages, the stopped-result guard holds (same-request stopped wake can't respawn, a fresh human request still can), and the before/after eval on gpt-5.6-luna is in my comment above. PR gate green.

AbirAbbas and others added 4 commits October 20, 2026 13:00
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Read current-turn person provenance and judge the newly consumed instruction, including a verbatim repeat of the stopped request.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Keep attempt snapshots and effective obligations distinct even when sibling tasks inherit the same request. Skip result-only cancelled work before classification, and show both readers only the independent eligible work with explicit cancellation boundaries.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Derive the stopped runtime flag from the persisted stopped ending. Keep restored cancelled results out of routing while ordinary failed results retain repair eligibility.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AbirAbbas and others added 2 commits October 20, 2026 13:42
…akes

Keep dev's routing inputs, floors and rulings whenever a human speaks, results
are mixed, or provenance is incomplete. Skip both classifiers only when every
delivered result is stopped for the current request in a result-only wake.

Remove scoped eligible-work questions, attempt tags and snapshot deduplication.
Retain frozen stop provenance and restored cancellation for the stop-only case.
Preserve untagged news through batching without changing existing owed asks.
Compare production wake, Submit, retry, cancel and recovery probes against dev,
and narrow the manual and change entry to the conservative guarantee.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ests

Both fresh-human stopped-wake tests submitted a steer from the answer hook but let the turn race the boundary drain, so CI could count one route reading instead of two. Drain inside the hook, as the mixed-result steering test already does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@AbirAbbas AbirAbbas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before merging I had two more independent review passes run on this, and they found edge cases in multi-result wakes. With a Steward principal and consent on:

  • A fresh human message consumed during a stopped-result wake was being suppressed.
  • One stopped result could veto an unrelated failed sibling's repair in the same wake.
  • A retried-then-stopped task could look like independent work.

Rather than keep adding conditions, I narrowed the stopped-result guard to the one case #1804 is about. It applies only when the wake is purely task results, every result is a stopped task's result for the current request (with positive provenance, no duplicate task IDs and no other news), and no fresh human message was consumed. In that case there's no classifier call and nothing starts. Every other wake routes exactly as dev does, and there are dev-compatibility tests for each scenario the reviews raised. A stopped task restored from a checkpoint also keeps its stopped flag now.

The prompt and manual changes separating "act now" from questions are unchanged. The manual and changelog promise only what the guard does. CI is green.

@AbirAbbas
AbirAbbas merged commit 1a5c885 into Agent-Field:dev Oct 20, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

session: auto-spawned tasks fire on questions/discussion, not just work orders (4x in one session)

3 participants