Repository navigation
Require explicit work orders before auto-starting tasks - #1845
Conversation
ZeroPoint95
left a comment
There was a problem hiding this comment.
@mikemikimike, the unwanted starts and stopped-task respawn reported in #1804 are worth fixing, and the stopped-result guard makes sense. I’m requesting a change to the routing rule before merging.
The shared prompt currently says: Every question is false, including "can we..." and "could you...". That treats being a question and being a request to act as mutually exclusive, even though both can be true:
- “Could you fix the failing tests?” is both a question and a request to act
- “Can you implement headless screenshots and add tests?” is likewise both a question and a clear request to act
- “How would headless screenshots work?” asks for an explanation
- “Can we get screenshots without a display?” is ambiguous without context and should not automatically start work
- “We should support screenshots eventually” expresses future direction, not authorization to begin
Please distinguish requests to perform work now from feasibility questions, discussion and brainstorming, rather than requiring a non-question grammatical form. Clear polite requests should remain eligible for normal task routing; ambiguous messages should stay in chat for clarification. Keep the stopped-result guard so stopping a task cannot itself restart the same request.
Please update both routing prompts and the manual consistently. The current authorization test checks that the rule’s text appears in the prompts, but doesn’t test how the classifier handles these examples. Add positive and negative examples covering the distinctions above, and provide a before/after classifier evaluation with the model and settings recorded. Preserve the regression showing that a stopped-result wake cannot respawn the task while a fresh human request remains eligible.
The goal is to prevent unauthorized work without suppressing ordinary ways users ask us to do things.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Separate feasibility questions from explanation requests in the shared route contract and the searchable chat manual. Pin examples with their complete classification clauses, and prove stopped-result suppression does not block fresh requests or normal remediation. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Give the stopped-result rule its own section so the manual context includes both the restart guard and the fresh-human-request exception. Pin the actual retrieved answer, since merely finding the long tasks page hid the rule beyond the section limit. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
@mikemikimike thanks for the fix and the follow-up commit. Since this had been waiting on another round, I've taken it over and pushed on top of your branch (no history rewritten). @ZeroPoint95, here's where each of your points landed: Requests to act vs. questions. 41a2e41 had already dropped the blanket "every question is false" rule. On top of that, feasibility ("Is headless screenshot capture possible on CI?") and explanation ("How would headless screenshots work?") are now separate examples in the shared contract, which both route prompts use, and in the manual. Authorization has its own heading in the manual, so search reaches it. The prompt test now ties each example to its full classification clause, so a stray "does not" elsewhere in the prompt can't satisfy it. Stopped-result guard. The production guard is unchanged. The regression now covers a same-request stopped wake (no classifier call, no new task), surrounding whitespace, a fresh human message with the same words (still eligible), an unrelated stopped task, and ordinary non-stopped remediation. With Before/after classifier eval. This used the real prompt constants from dev (26662f1) and this head,
There were 10 transport timeouts across the 180 calls. They're excluded and not counted as correct. The one remaining miss is the pre-turn read of "Could we clean up the configuration loader across all packages?" (ambiguous), which the post-turn read rejects all three times. The settings and raw results can be shared if you want to rerun it.
|
AbirAbbas
left a comment
There was a problem hiding this comment.
Verified at 01b4701: both route prompts and the manual separate requests to act now from feasibility/explanation/ambiguous/future-direction messages, the stopped-result guard holds (same-request stopped wake can't respawn, a fresh human request still can), and the before/after eval on gpt-5.6-luna is in my comment above. PR gate green.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Read current-turn person provenance and judge the newly consumed instruction, including a verbatim repeat of the stopped request. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Keep attempt snapshots and effective obligations distinct even when sibling tasks inherit the same request. Skip result-only cancelled work before classification, and show both readers only the independent eligible work with explicit cancellation boundaries. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Derive the stopped runtime flag from the persisted stopped ending. Keep restored cancelled results out of routing while ordinary failed results retain repair eligibility. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…akes Keep dev's routing inputs, floors and rulings whenever a human speaks, results are mixed, or provenance is incomplete. Skip both classifiers only when every delivered result is stopped for the current request in a result-only wake. Remove scoped eligible-work questions, attempt tags and snapshot deduplication. Retain frozen stop provenance and restored cancellation for the stop-only case. Preserve untagged news through batching without changing existing owed asks. Compare production wake, Submit, retry, cancel and recovery probes against dev, and narrow the manual and change entry to the conservative guarantee. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ests Both fresh-human stopped-wake tests submitted a steer from the answer hook but let the turn race the boundary drain, so CI could count one route reading instead of two. Drain inside the hook, as the mixed-result steering test already does. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
AbirAbbas
left a comment
There was a problem hiding this comment.
Before merging I had two more independent review passes run on this, and they found edge cases in multi-result wakes. With a Steward principal and consent on:
- A fresh human message consumed during a stopped-result wake was being suppressed.
- One stopped result could veto an unrelated failed sibling's repair in the same wake.
- A retried-then-stopped task could look like independent work.
Rather than keep adding conditions, I narrowed the stopped-result guard to the one case #1804 is about. It applies only when the wake is purely task results, every result is a stopped task's result for the current request (with positive provenance, no duplicate task IDs and no other news), and no fresh human message was consumed. In that case there's no classifier call and nothing starts. Every other wake routes exactly as dev does, and there are dev-compatibility tests for each scenario the reviews raised. A stopped task restored from a checkpoint also keeps its stopped flag now.
The prompt and manual changes separating "act now" from questions are unchanged. The manual and changelog promise only what the guard does. CI is green.
Summary
Validation
go build ./...passed.gofmt -d internal/session/route_judge.go internal/session/route_judge_test.goproduced no diff.go vetover the Windows production sources ininternal/sessionpassed.go generate ./internal/manualpassed.go test -count=1 ./cmd/codeaf-changespassed.go test -count=1 -p 2 -timeout=15m ./...fails on existing Windows-incompatible test sources and platform assumptions; the same command also fails on cleandev.internal/sessiontest package cannot build on Windows because existing tests depend on Unix-only packages and syscalls.Fixes #1804
#1804