From 7c15eef0bccbac5e483cc951bf51dc4a85546311 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 18 Aug 2026 19:57:37 +0000 Subject: [PATCH] review: verdicts record a disposition per lifted claim MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The change the 2026-08-18 falsified-by efficacy review recommended (agent_failure_modes.md item 6 Outcome): across 22 ship gates the ledger could not distinguish a healthy adversarial claim pass from a skipped one — both wrote a bare 'review CLEAN'. Now, when the ReviewSurface lifts any 'claims to falsify', the verdict carries one disposition line per claim (basis-cited / idle / FINDING unverified-claim), written by the reviewer at verdict time; a bare CLEAN over a non-empty claims surface is malformed evidence. An empty surface requires nothing, so the 74-95% of ships that lift no claims gain no busywork. Surfaces: faculty AGENTS.md step 2a + verdict mapping; _review.py human-emit epilogue (printed only when claims lifted); AUTONOMY.md autonomous-ship-gate review leg; ship_library reference.md gate evidence line. No trigger-vocabulary change (measured as neither empty nor saturated). 2 new pinning tests; suite 351 passed. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01WH4NizvBK2jki2Uh5TMABh --- AUTONOMY.md | 5 ++++- agents/faculties/review/AGENTS.md | 14 ++++++++++++-- agents/faculties/review/_review.py | 6 ++++++ docs/agent_failure_modes.md | 8 +++++--- skills/ship_library/reference.md | 3 ++- tests/test_review_claims.py | 29 +++++++++++++++++++++++++++++ 6 files changed, 58 insertions(+), 7 deletions(-) diff --git a/AUTONOMY.md b/AUTONOMY.md index 1498faa..fd8f06f 100644 --- a/AUTONOMY.md +++ b/AUTONOMY.md @@ -158,7 +158,10 @@ applicability rule so "n/a" is a stated fact, never an assumption: feel stronger. 3. **Review** — review-faculty verdict **CLEAN** (`agents/faculties/review/AGENTS.md`). FINDINGS → resolve and re-review, or - park to a human checkpoint; BLOCKED → park. + park to a human checkpoint; BLOCKED → park. When the surface lifted any + `claims to falsify`, CLEAN carries one disposition line per claim + (basis-cited / idle / finding — faculty AGENTS.md step 2a); a bare CLEAN + over a non-empty claims surface is malformed evidence, not CLEAN. 4. **Heart** — verdict **GREEN** or **STALE**, or **YELLOW whose reason set is contained in the set the human acknowledged at launch**. Heart observes organism state, not the branch (the audit confirmed its legs never see diff --git a/agents/faculties/review/AGENTS.md b/agents/faculties/review/AGENTS.md index 92998be..47aaf43 100644 --- a/agents/faculties/review/AGENTS.md +++ b/agents/faculties/review/AGENTS.md @@ -62,11 +62,21 @@ Heart and the agent reasons over the verdict. category `unverified-claim` — the author asserted an effect they did not show. Scope it to genuinely load-bearing claims (an effect that, if wrong, ships a bug); an idle turn of phrase is not a finding — say so and move on, - so the pass does not decay into rote noise. + so the pass does not decay into rote noise. **Record a disposition per + claim**: the verdict carries one line per lifted claim — + `claim: "" → basis-cited: | idle | FINDING (unverified-claim)` + — written by the reviewer at verdict time, never by the author. This is + what makes the pass auditable: the 2026-08-18 efficacy review + (`docs/agent_failure_modes.md` item 6 Outcome) found that across 22 ship + gates a healthy pass and a skipped one wrote the identical ledger row. + An empty surface requires nothing. 3. Map the outcome to the verdict: any unresolved must-fix → **FINDINGS** (ranked list, file:line, failure scenario) — including any `unverified-claim` from step 2a; nothing → **CLEAN**; could not - complete steps 1–2 → **BLOCKED** (say why). + complete steps 1–2 → **BLOCKED** (say why). A CLEAN over a non-empty + `claims to falsify` surface **must** carry the step-2a disposition lines — + without them the evidence is malformed, not CLEAN (the ship-checkpoint + reader can see the omission; that is the reader-enforcement). ## Run diff --git a/agents/faculties/review/_review.py b/agents/faculties/review/_review.py index 55a1aac..8ecc93b 100755 --- a/agents/faculties/review/_review.py +++ b/agents/faculties/review/_review.py @@ -190,6 +190,12 @@ def emit_human(surfaces: list[dict]) -> None: print("file:line, failure scenario) | BLOCKED (could not review — say why).") print("A load-bearing claim above with no falsified-by basis in the branch is") print("a FINDING (unverified-claim) — see the faculty AGENTS.md.") + if any(s.get("claims_to_falsify") for s in surfaces): + print("Record ONE disposition line per lifted claim in the verdict:") + print(' claim: "" -> basis-cited: ' + " | idle | FINDING (unverified-claim)") + print("A CLEAN over a non-empty claims surface without dispositions is") + print("malformed evidence, not CLEAN (faculty AGENTS.md step 2a).") def main(argv=None) -> int: diff --git a/docs/agent_failure_modes.md b/docs/agent_failure_modes.md index 1ab2ec6..480980f 100644 --- a/docs/agent_failure_modes.md +++ b/docs/agent_failure_modes.md @@ -179,9 +179,11 @@ Each: catalogue entries caught → why it fires at the decisive moment → cost agent's verdict gains a one-line disposition per lifted claim (basis-cited / idle / finding) recorded in the ship evidence, so a rote pass becomes visible ledger drift per this doc's own ranking (detecting - beats reminding). Filed: PyAutoMind - `draft/feature/pyautobrain/review_claim_dispositions.md`; full numbers in - PyAutoMind `complete/2026/08/falsified-by-checkpoint-efficacy-review.md`. + beats reminding). Implemented 2026-08-18: faculty AGENTS.md step 2a + the + surface epilogue + the AUTONOMY.md review leg + the ship evidence format + (record: PyAutoMind `complete/2026/08/review-claim-dispositions.md`); full + numbers in PyAutoMind + `complete/2026/08/falsified-by-checkpoint-efficacy-review.md`. ## 6. The memory system, attacked honestly diff --git a/skills/ship_library/reference.md b/skills/ship_library/reference.md index b82d6c2..eb61755 100644 --- a/skills/ship_library/reference.md +++ b/skills/ship_library/reference.md @@ -62,7 +62,8 @@ what the human validates instead of the pre-approval they didn't give: - Effective level: safe (header: , cap: → ) - Plan: on the issue (#), written at start, unmodified since - Gate: tests · smoke · - review CLEAN · Heart + review CLEAN <+ one disposition per lifted claim, when the surface lifted + any — faculty AGENTS.md step 2a> · Heart - [ ] Human: plan sound in hindsight? - [ ] Human: diff matches plan (no scope creep)? - [ ] Human: merge, amend, or reject — then log the outcome diff --git a/tests/test_review_claims.py b/tests/test_review_claims.py index 5438453..a61005d 100644 --- a/tests/test_review_claims.py +++ b/tests/test_review_claims.py @@ -58,3 +58,32 @@ def test_verified_and_safe_to_delete_are_claims(): claims = load_bearing_claims(text) j = " ".join(claims).lower() assert "verified" in j and "safe to delete" in j and "zero diff" in j + + +def test_emit_human_demands_dispositions_when_claims_lifted(capsys): + from _review import emit_human + + surface = { + "repo": "PyAutoDemo", "path": "/tmp/x", "branch": "feature/x", + "base": "abc123def456", "commits_ahead": 1, "commits": ["abc fix"], + "shortstat": "1 file changed", "files": ["M\tf.py"], + "risk_flags": [], "claims_to_falsify": ["This change is a no-op for CI."], + } + emit_human([surface]) + out = capsys.readouterr().out + assert "ONE disposition line per lifted claim" in out + assert "malformed evidence" in out + + +def test_emit_human_no_disposition_demand_on_empty_surface(capsys): + from _review import emit_human + + surface = { + "repo": "PyAutoDemo", "path": "/tmp/x", "branch": "feature/x", + "base": "abc123def456", "commits_ahead": 1, "commits": ["abc fix"], + "shortstat": "1 file changed", "files": ["M\tf.py"], + "risk_flags": [], "claims_to_falsify": [], + } + emit_human([surface]) + out = capsys.readouterr().out + assert "disposition" not in out.lower()