Batch workflow: the review-cost model, the unattended ship gate, and the Batch Agent - #319
Merged
Merged
Conversation
Difficulty measures blast radius. It cannot answer the question a batch has to
be planned against, which is what a task costs the HUMAN once it lands. Four
read-only outputs on the sizing faculty now answer it:
Consequence: notify | glance | judge — how much review the work needs
Witness: free text — the machine-checkable claim that
makes it reviewable
Review-minutes: integer — a seed, not a measurement
Unattended: ready | needs-slicing | never
Graded by rules over repo class (from repos.yaml, where repo identity is already
declared once) and over surface, never by an agent's reading of its own work —
the ledger's base rate for an agent mis-scoping its own change is 20% (68 of 332
records in 2026-08 carry a correction or a retraction), so self-assessment is
not an input.
THE RULE THAT CARRIES THE MODEL: no Witness means judge. Without it the field
would be aspirational — a prompt could claim a cheap tier while offering the
reviewer nothing but the diff. With it, choosing a cheap tier means committing
at conception to producing evidence, which is what actually makes work
reviewable in minutes.
Measured over the 153 backlog prompts today: 151 grade judge, because 3 carry a
witness. Given one, the same backlog grades 33 notify / 104 glance / 16 judge.
The whole distance between "everything costs a PI's hour" and "a fifth of it
costs nothing" is whether prompts declare what will make them checkable. That
number is the case for the field, and it is in the faculty's AGENTS.md.
Unattended is deliberately not difficulty renamed: needs-slicing keys off the
compaction rule (a task that would need context compaction to finish is too big
to run unattended), which is measured rather than cautious. A single-repo large
task still grades ready; a large one across four repos does not.
Precedence is the module's existing rule, extended: declared beats derived, with
the derived value returned alongside so the split is reported rather than
silently resolved. One case is called out in the surface rather than hidden —
a prompt the heuristic reads too-large and the author calls medium still grades
ready, but says so.
Known limit, documented rather than hidden: the judged-surface test is keyword
matching. Fenced and inline code are masked (a prompt quoting a surface is
documenting it, not touching it — the same rule declared_header applies), but
prose describing a surface still trips it, and the prompts specifying this model
are the worst offenders. The error is in the safe direction, and the keyword
list is kept narrow because a loose one does not fail safe: it grades everything
judge and the model stops discriminating.
10 new tests including a golden pin; all three load-bearing guards were
confirmed to FAIL against a deliberately broken tree before being trusted.
654 pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RQeMJZznA3xTQXX4PqWg1v
…gger Intake now derives and writes Consequence:, Review-minutes: and Unattended: at conception, and reads a Witness: when the author declares one. Witness is the one field nothing derives: an invented one is plausible prose with nothing behind it, which is worse than none, because the value of the field is that its absence is informative. So it is kept OUT of HEADER_FIELDS — `intake formalise` writes every field it finds missing there, and would have auto-invented one across the whole backlog. Consequence and Review-minutes are derived and do join the hygiene set. When no witness is declared, the IntakeDecision says so and explains what it costs: the prompt grades judge, a PI's quarter-hour, whatever its size. infer_autonomy no longer returns supervised on repo_count > 1. Repo count is blast radius, already priced at +2 per repo in estimate_difficulty; this field is supposed to encode whether a human's judgement is needed, and a change across four repos mechanically needs no more of it than the same change in one. TWO CORRECTIONS TO THE PLAN THIS IMPLEMENTS, both measured here. The epic asserted that this one rule caused 120 of 137 prompts to read supervised. It does not. Re-deriving all 153 draft prompts: dropping the trigger takes safe 30 -> 55 and supervised 117 -> 92, and repo_count > 1 is the SOLE trigger for 25 prompts — the largest single one, ahead of large-or-above (20) and architectural risk (17), but nothing like 120. Those 120 are declared levels written by earlier intake runs, and the triggers overlap heavily. The change is right on its merits; it is not the unblocking it was taken for, and where that actually lives is the ship-sign-off change in phase 3 — 19 of the 46 parked rows are the contract park supervised imposes at ship, which no grading change touches. The first draft replaced repo_count with human_judgement, on the reasoning that ambiguity is what predicts a park. Measured, it made things worse: safe fell to 24, because the ambiguity keywords fire on 63% of prompts and catch well-written ones indiscriminately. Same mistake as the rule it replaced — a loose proxy standing in for a judgement it does not measure. Reverted, and locked by a test so nobody re-proposes it from first principles. AUTONOMY.md carries the change as a DATED EXPERIMENT, not a graduation: 20 unattended launches, the adversarial review leg mandatory (so the window does not start until phase 3 exists), rows per work-type, and a new human-stamped `rejected-at-review` outcome — added because `rejected` has never been used in 238 rows, having been routed around rather than earned, and a demotion trigger nothing can pull is not a safety device. 7 new tests, 661 pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RQeMJZznA3xTQXX4PqWg1v
…o witness The pick list was `difficulty == small and autonomy == safe`. Ten prompts in the live backlog carried `safe`, so the surface whose whole job is handing out unattended work had almost nothing to hand out. It now selects on `Unattended: ready` and orders by review-minutes ASCENDING — the list is read when the human has a slot to fill and wants to know what fits in it, and `Highest priority` above is where importance is answered. Against the re-graded backlog it goes from near-empty to 71. An ungraded prompt sorts last rather than disappearing, which is the page's standing rule for unknowns. New hygiene row: prompts with no `Witness:`. Not an error — a prompt without one grades `judge` by design, and that default is what makes the field bite. But nothing derives or backfills a witness (an invented one is plausible prose with nothing behind it), so the only thing that can clear the row is a human writing one, and the page has to say which prompts are waiting. 150 of 153 today. The census record carries consequence / witness / review_minutes / unattended. The old `test_quick_wins_are_small_and_safe_only` is replaced rather than patched — the behaviour deliberately changed — by four tests covering the selector, the ordering, the ungraded-sorts-last rule and the hygiene row. 664 pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RQeMJZznA3xTQXX4PqWg1v
…se 3) Four doctrine changes, each dated, each citing its evidence, each with an explicit revert condition. The gate was designed for runs with a human in the session, and every leg has an assumption that breaks at 3am. 1. WHAT A BATCH LAUNCH IS. Activation says levels bind only on an explicit --auto, "never ambient: no config flag, no environment variable, no remembered mode". A wave firing at 4am under an approval given at 17:00 is a stored grant — the shape the corrective-PR section already voids. So: a batch dispatch is ONE launch; membership is fixed at approval; the grant expires with the shift; the terms are written into the batch record; and the HUMAN performs the dispatch. A scheduler may carry the timing, never the authority. 2. LEG 4 UNDER A BATCH LAUNCH. YELLOW passes only where the reason set is the one acknowledged at launch, and nobody can acknowledge anything overnight — so a 15-hour shift would ride on that set staying frozen. It does not stay frozen for an afternoon: the log shows a drift count going 2 -> 4 -> 6 across one day, and a benign new reason appearing mid-session from a run's own sibling merge, which in a batch means wave 1 can park waves 2 and 3. The pressure has already produced a violation rather than a park — cmap-magma-default shipped under a STANDING ack, which doctrine voids. So for a batch: the human acks a named reason set for the shift; a run parks on new RED, or a new YELLOW whose repo intersects its own; a reason generated by an earlier member is named and does not park later ones; the grant expires with the shift. RED still parks everywhere, and YELLOW is still never acknowledged autonomously — this only defines how far a human's acknowledgement reaches. 3. LEG 5, THE INDEPENDENT ADVERSARY. Required for batch launches and for the multi-repo autonomy experiment. A second reading by a DIFFERENT MODEL whose job is to falsify the change's claims, the Witness: first. Step 2a was already adversarial in procedure and was still run, in practice, by the branch's own author: the efficacy review found "a healthy pass and a rote one write the identical ledger row", and that the one confirmed-wrong claim of its window lived outside the surface the stage reads. A self-run adversary leg is an ABSENT leg, not a weak one, and recording it as run is a false ledger row. Implemented as `review --witness ... --adversary`. 4. DECIDE-AND-FLAG, capped. Park-and-ask costs a whole shift when the human is not one message away. So a batch run may take the more reversible option and record it — but at most ONE per PR, stating the rejected alternative and the one-command revert (if it cannot write the revert, it was not reversible), never on a public API, default, error contract or external reporter's file, and never on a judge-tier task. The base rate this is sized against is measured: 68 of 332 August records carry a correction or retraction. Also reconciled: the gate is four legs, five under a batch launch — phrasing updated in AUTONOMY.md and both ship skills rather than renaming the gate, since ordinary --auto runs are unchanged. Calibration row gains an adversary column. 6 new tests, 670 pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RQeMJZznA3xTQXX4PqWg1v
…s 1-2)
`pyauto-brain batch plan` composes one unattended shift and emits a
BatchDecision. It PROPOSES; it never dispatches — approving it in a slot is what
launches a batch (AUTONOMY.md, "What a batch launch is"), so the schedule can
carry the timing but never the authority.
The composition rule is a budget in REVIEW-MINUTES, not a task count:
sum(Review-minutes) over glance and judge members <= the slot's budget, default
45. Read against the ledger an honest hour holds about three library-touching
tasks, so planning by count over-promises capacity roughly threefold and lands
the overflow on the human at 6am. Work above the budget is the FILL — zero
review-minute work, sized by the token allowance rather than the human's hour,
which is what lets the whole weekly budget be spent without growing the queue.
Constraints, each of which states itself in `rejected` (a planner that silently
drops work teaches the human to distrust its numbers):
- one member per LIBRARY repo per shift. They do not collide at dispatch —
separate worktrees — they collide at merge, when the first /prm moves main
and invalidates the others' test and smoke evidence. Workspace, docs and
organ repos are exempt: two docs changes in one shift cost nothing.
- one slice per epic (phases are ordered, so they could not parallelise
anyway, and this is what interleaves epics with standalone work)
- Unattended: ready only; Blocked-by: excluded
- lane match, with the other lane's ready count REPORTED rather than dropped
Backpressure ramps and never deadlocks, counted in tasks awaiting review rather
than PRs (94 of 332 August records named two or more PRs, so a PR cap trips on
one healthy batch): clear -> full budget; above half -> the review-bearing half
halves, the fill does not; at the cap -> the floor, fill only, dispatched
whether or not the human turned up. An EMPTY floor is reported as a finding, not
a deadlock — it means nothing in the backlog costs zero review-minutes, which is
what the notify tier and the Witness field exist to change.
Lane detection is probed from the environment, never declared: a session that
could be told where it is could plan local-dev work it cannot run. It reads the
signal the organism already uses — a remote session has no gh.
Also: the Lane: any | local-dev header in the sizing faculty and intake, spelled
in WORKFLOW.md's existing environment vocabulary rather than a parallel
cloud/laptop one.
Against the live backlog the planner picks 2 members for 40 of 45 minutes — the
honest capacity while 150 of 153 prompts carry no witness and therefore grade
judge at 20 minutes each. That number is the point, not a defect.
12 new tests, 682 pass. (Pre-existing and untouched: skills/prm exceeds the
200-line skill budget.)
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RQeMJZznA3xTQXX4PqWg1v
The first version drove the CLI against this checkout with `--repo`, asserting on its stdout. That passed locally and failed in CI with four empty-output errors, because a CI checkout has no `origin/main` to diff: `repo_surface` returns None for every repo, `main` exits 4 with an empty stdout, and every assertion about the contract text failed for a reason unrelated to the contract. Reproduced before fixing — a fresh `git init` with no origin/main gives "review: no reviewable diff against origin/main", exit 4, stdout empty. Fixed at the seam rather than by working around it: `surface_payload()` is split out of `main` as a pure assembly function, so the JSON contract can be tested without git, and the human-output tests call `emit_human` with a synthetic surface. The fixture mirrors `repo_surface`'s return shape exactly and was checked key-for-key against the real producer. The lesson is the one the tests are about: a test whose subject is a block of contract text must not depend on the environment having a diff. 683 pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RQeMJZznA3xTQXX4PqWg1v
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phases 0a, 0b, 0c, 1, 2 and 3 of the
two-slot-batchingepic (PyAutoMind/draft/feature/pyautomind/two_slot_batching_epic.md). Paired with PyAutoMind's ledger half.Why this shape
The epic was filed to move from one babysat task per chat to a bounded human slot with everything else unattended. Reading the ledger while building it changed the design: there is no throughput problem. August 2026 shipped 332 completion records, about eleven a day. The scarce resource is the human's judgement, and a batch layer that plans by task count spends it faster rather than slower.
So the work here is about reducing judgement per merged unit, not scheduling more of it.
What landed
The review-cost model (sizing faculty). Four read-only outputs beside
Difficulty:, which measures blast radius and cannot answer what a task costs the human:Consequence:(notify/glance/judge),Witness:(the machine-checkable claim that makes it reviewable),Review-minutes:(a seed, explicitly not a measurement),Unattended:(ready/needs-slicing/never, keyed off the compaction rule rather than off size).Graded by rules over repo class and surface, never by an agent's reading of its own work — the ledger's base rate for an agent mis-scoping its own change is 20% (68 of 332 August records carry a correction or a retraction).
The load-bearing rule: no
Witness:meansjudge. Measured over the 153 backlog prompts, 151 gradejudge, because three carry a witness. Given one, the same backlog grades 33notify/ 104glance/ 16judge. The whole distance between "everything costs a PI's hour" and "a fifth of it costs nothing" is whether prompts declare what will make them checkable.infer_autonomydrops therepo_count > 1trigger. Repo count is blast radius, already priced at +2 per repo inestimate_difficulty.The ship gate, fixed for unattended conditions (
AUTONOMY.md, four dated changes, each with a revert condition): what a batch launch is; leg 4's shift-scoped Heart acknowledgement; leg 5, the independent adversary; and decide-and-flag capped at one per PR.The Batch Agent —
pyauto-brain batch plan. Composes one shift against a review-minute budget, states every rejection with its reason, ramps backpressure rather than deadlocking, and proposes rather than dispatches.The dashboard's pick list — "Quick wins" (
small AND safe, near-empty at ten candidates) becomes "Fits a slot" (Unattended: ready, cheapest-to-review first): 71 against the re-graded backlog.Two corrections to the plan, both measured here
The epic's headline claim was wrong. It asserted
repo_count > 1caused 120 of 137 prompts to readsupervised. Re-deriving all 153: dropping it takessafe30 → 55 andsupervised117 → 92, and it is the sole trigger for 25 prompts — the largest single one, ahead oflarge-or-above (20) and architectural risk (17), but nothing like 120. Those 120 are declared levels from earlier intake runs and the triggers overlap heavily. Corrected in the epic, in the phase prompt and inAUTONOMY.md.The first fix made it worse. Replacing
repo_countwithhuman_judgementtooksafedown to 24 — the ambiguity keywords fire on 63% of prompts and catch well-written ones indiscriminately. Same mistake as the rule it replaced. Reverted, and locked by a test so it is not re-proposed from first principles.Two things deliberately not done
Witness:is out ofHEADER_FIELDS. That set drivesintake formalise, which writes every missing field in place — including it would have auto-invented a witness across the whole backlog. Plausible prose with nothing behind it is worse than none, because the value of the field is that its absence is informative. The dashboard reports the gap as hygiene instead (150 prompts).The gate is not renamed. It is four legs for ordinary
--autoruns and five under a batch launch; the phrasing says both rather than sweeping "four-leg" everywhere and making unchanged runs look changed.Validation
682 tests pass (35 new). The three load-bearing guards in the review-cost tests were confirmed to fail against a deliberately broken tree before being trusted, per the standard
complete/2026/08/mobile-performance-review.mdsets.Known and pre-existing, untouched by this branch:
skills/prm/exceeds the 200-line skill budget (bin/check_skill_line_counts.sh), and PyAutoMind'stest_session_bootstrap.py::test_the_hook_survives_an_unwritable_tools_dirfails in a root container.Not in this PR
Epic phases 4 (the
notifymerge tier — shadow window open until 27 Sep), 5 (dispatch), 6 (the batch board), 7 (budget) and 8 (the laptop lane).batch sliceandbatch collectare stubs of intent in the conductor'sAGENTS.md;planis useful alone.Generated by Claude Code