From 3e9302ce51108473b5cae70068c9f3133f4806c7 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 03:17:58 +0000 Subject: [PATCH 1/8] new(ready): READY only via bin/ready.sh, with its evidence committed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit bin/ready.sh runs gate-check Stage 5 (Stage 6 from the register's position), the look ledger, the must-have list when present and, under Interview mode auto, an independent reviewer's scored verdict file. K of N per check; exits non-zero unless all pass; writes the READY line itself and commits docs/progress/ready--.md with explicit paths. look-ledger.sh summary --write puts the git-ignored ledger into a tracked docs/progress/look-summary.tsv (ready.sh, close-task.sh); gate-check Stage 5 reads it when the ledger is absent, so a copy of the repo FAILs, not MANUAL. Field evidence: two unattended field runs, 2026-11 — READY hand-written with 0 of ~78 pages looked at; another kit marked ready 3 hours early, no review. Exercised against a project rebuilt from the captured lab-app pages and screenshots (tests/wave2/fixtures/look-shots), both layouts. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y --- CHANGELOG.md | 1 + README.md | 1 + ROUTING.md | 1 + agents/architect-agent.md | 1 + agents/ba-agent.md | 1 + agents/gate-agent.md | 1 + agents/mdl-agent.md | 1 + agents/review-agent.md | 1 + agents/test-agent.md | 1 + bin/gate-check.sh | 30 ++-- bin/init-project.sh | 2 + bin/lib/skill-routing.tsv | 1 + bin/ready.sh | 274 +++++++++++++++++++++++++++++++++++++ project-bin/close-task.sh | 10 ++ project-bin/look-ledger.sh | 53 ++++++- skills/dispatching.md | 9 ++ tests/wave2/test-ready.sh | 144 +++++++++++++++++++ 17 files changed, 521 insertions(+), 11 deletions(-) create mode 100755 bin/ready.sh create mode 100755 tests/wave2/test-ready.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 105d7619..de9f1e02 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m `docs/RELEASE-PROCESS.md` §8. ## Unreleased +- new(ready, look-ledger, gate-check): **READY is earned through one command, and its evidence is in git.** `bin/ready.sh [--module M | --app A]` runs `gate-check.sh` Stage 5 (Stage 6 once the register is there), the look ledger, the must-have list when present and, under `Interview mode: auto`, an independent reviewer's verdict file (`docs/progress/look-reviews/`, every page scored 1–5 after its last build, below 4 blocks); it prints K of N per check, exits non-zero unless all pass, writes the READY line to `docs/progress/ready.log` itself and commits `docs/progress/ready--.md` with explicit paths. `look-ledger.sh summary --write` puts the git-ignored ledger into a tracked `docs/progress/look-summary.tsv` (written by `ready.sh` and every `close-task.sh`), and `gate-check.sh` Stage 5 reads it when the ledger is absent, so a copy of the repo reports FAIL instead of MANUAL. `dispatching.md` gains the reviewer dispatch rule; the `ready` routing row reaches `CLAUDE.local.md` and every agent stub (field runs, 2026-11: a build wrote READY by hand with 0 of ~78 pages looked at; another marked itself ready 3 hours early with no review pass) — two unattended field runs - fix(status, agents): **`bin/status.sh` WATCHes an Agent Editor project whose extension module does not travel with the model (#778)**: Studio Pro reads `agenteditor.agent` from `modules/AgentEditor.mxmodule` beside the `.mpr`, and a sync of only `.mpr` + `mprcontents/` left it out of git, so the builder stayed green while every colleague's clone showed `unknown type 'agenteditor.agent'`. Two WATCH items, warn-only (exit code and every verdict unchanged), local files and `git ls-files` only, root and `app/` layouts: a `modules/*.mxmodule` git does not track, and a project using Agent Editor (`AgentEditorCommons` in `analysis/app-facts/inventory.json`, or a `create agent` statement in `mdlsource/`) with no `modules/AgentEditor*.mxmodule`. `skills/mendix-agents.md` (on demand) gains the rule plus the note that the GenAI key, `Encryption.EncryptionKey` and `GenAICommons.StoreTraces` belong in the Default configuration, not a start script. Empty agent references / missing model version (mendixlabs/mxcli#1867) and unset runtime constants are not checked: they need a captured agent document (option 2). Fixture `tests/wave2/test-status-extension-modules.sh` — field report from an Agent Editor project, Nov 2026 - fix(permissions): **exec approval `auto` no longer prompts on every exec's mode check, and `install-claude-hooks.sh` no longer runs unprompted (#782)**: `install-claude-permissions.sh` now allow-lists `exec-approval.sh`, `interview-mode.sh` and `permission-profile.sh`, so their reads run without a prompt, while their only writers, `--set` and `--clear`, prompt through six `ask` rules (`Bash(*exec-approval.sh*--set*)`, …) in the shared settings under every profile (an ask rule beats an allow rule, `full`'s bare `Bash` included). `install-claude-hooks.sh`, which writes `~/.claude/settings.json` for every repo on the machine, gets no allow rule. A re-install adds the ask rules and removes the hooks-installer rule an earlier install added (sidecar); entries someone else wrote are untouched, and `sync-project.sh`'s permission check names both until then. Field run on claude 2.1.325: the read ran, `--set auto` and `--clear` were refused, and with the ask rules removed `--set` ran — from the two independent reviews of #781 - process(skills): **trim PR, moves only: always-loaded words 66,271 -> 64,984 (budget 80,000); Stage 5 test 21,906 -> 20,619, gate 22,954 -> 21,744, review 31,681 -> 30,394 (#783)**: `testing-shape.md` §5's stack and scratch-copy mechanics move verbatim to `e2e-harness-base.md` § Bringing the stack up (the rule and a one-line list of its rules stay in §5); the three new Unit false-green rows move to `unit-test.md` § False-green rows (a pointer row stays in the register); `module-review.md`'s re-shoot mechanics move to `iterative-build-loop.md` § Pipeline mode (the rule, the denominator and the failure example stay) and its MXTK-AI001 sentence to `marketplace-catalogue.md` § Reviewing a built module. Every removed line is in its destination, checked line by line — weekly word-budget audit (#783) diff --git a/README.md b/README.md index 9da2cf91..fa9e6378 100644 --- a/README.md +++ b/README.md @@ -914,6 +914,7 @@ Then filter by Role(s). A helper dispatched with a role stub (ba, architect, mdl | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | `bin/exec-approval.sh` | 5,6 | mdl, gate | | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | `skills/testing-shape.md` | 5,6 | test, gate, review | | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | `project-bin/verify-module.sh` | 5,6 | mdl, gate, test, review | +| Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | `bin/ready.sh` | 5,6 | every role | | A CSS/SCSS or style change that appears to have done nothing (a colour or class that won't show), a focus ring or outline that paints on every page landing or load, or an app still grey after a design port every instrument called green — the three ways a correct rule paints nothing (matches nothing / matches chrome / loses the cascade), the two reads that tell them apart, and the class that arrived in the stylesheet and is bound to no widget | `skills/learned-css-that-never-applied.md` | 5,6 | review, gate (mdl: on demand) | | Before trusting a green check/exec/DESCRIBE result as proof, or when a runtime symptom appears over a fully green model — the register of constructs that pass early rungs and fail later ones | `skills/learned-detection-gaps.md` | 5,6 | gate, review (mdl: on demand) | | Creating any entity, or calling a module security-ready — entity and grants land in one script, and ready means SHOW SECURITY MATRIX proves it | `skills/security-is-not-a-later-script.md` | 5 | gate, review (mdl: on demand) | diff --git a/ROUTING.md b/ROUTING.md index 5fe4877d..8c256c72 100644 --- a/ROUTING.md +++ b/ROUTING.md @@ -225,6 +225,7 @@ picks the row up. That is the whole procedure — there is no second list to rem |---|---|---|---|---| | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | `skills/testing-shape.md` | test,gate,review | 5,6 | baseline | | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | `project-bin/verify-module.sh` | mdl,gate,test,review | 5,6 | baseline | +| Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | `bin/ready.sh` | all | 5,6 | baseline | | Auditing or regression/e2e-testing an EXISTING app — no intake, no stages, no gates | `skills/existing-app-assurance.md` | test,review | 6 | ondemand | | Running the ledger rung alone — recompute every stored ledger status against the live model and catch the STALE rows that claim built for something the model no longer has | `project-bin/conformance-check.sh` | gate,review | 5,6 | ondemand | | Running the wiring rung alone — a module imported but never reached, an element built but wired to nothing, a boundary crossed; mxbuild and e2e are blind to all three | `project-bin/graph-sweep.sh` | review,gate | 5,6 | ondemand | diff --git a/agents/architect-agent.md b/agents/architect-agent.md index c357020c..58168c09 100644 --- a/agents/architect-agent.md +++ b/agents/architect-agent.md @@ -41,6 +41,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/checkpoints/checkpoint-template.md` | Any stage transition — the 2+1 format every CAC uses, and the one-register rule (answers land in PROJECT.md, never in a separate state file). The seven CACs themselves are routed per stage in the situational table | | `skills/module-folder-convention.md` | Placing any document in a module — before the first `create`. Feature group, then Pages/Microflows/Services/Resources; the path comes from the brief's folder plan, and the table says which types mxcli can actually place | | `project-bin/check-design-reaches-app.sh` | After the FIRST build that follows any design-system port, and before any page is built on it — reads the BUILT stylesheet and reports how many framework knobs point at a design token, how many tokens arrived, how many component classes arrived, each with its denominator. Measured on a real run: 55 tokens ported correctly into the right file, 0 of 35 knobs bound and 0 of 20 classes present, two build phases shipped in the framework's default blue with mx check, mxcli lint, the MDL suite and two e2e journeys all green | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/status.sh` | The first command of every session, and any time someone asks "where are we" or "what next" — one screen: stage, done/overdue, the ONE next action, from the instruments, never from memory | | `skills/retesting-learned-rules.md` | Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | diff --git a/agents/ba-agent.md b/agents/ba-agent.md index 1b9d0d4b..1c14f9e4 100644 --- a/agents/ba-agent.md +++ b/agents/ba-agent.md @@ -53,6 +53,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `bin/question-kinds.sh` | Deciding who answers a question — before putting any batch to the user. gap/conflict/choice/user-only is what keeps a gate batch at four questions instead of 127 | | `bin/facts-lock.sh` | Writing BRDs, especially several in parallel — "build" before the fan-out, "check" before any BRD is called done | | `skills/module-brief.md` | Building any module — before the first script. The mdl-agent's single per-module input | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/status.sh` | The first command of every session, and any time someone asks "where are we" or "what next" — one screen: stage, done/overdue, the ONE next action, from the instruments, never from memory | | `skills/retesting-learned-rules.md` | Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | diff --git a/agents/gate-agent.md b/agents/gate-agent.md index 2563531f..f1608ee8 100644 --- a/agents/gate-agent.md +++ b/agents/gate-agent.md @@ -40,6 +40,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `bin/exec-approval.sh` | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | | `skills/testing-shape.md` | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `skills/learned-css-that-never-applied.md` | A CSS/SCSS or style change that appears to have done nothing (a colour or class that won't show), a focus ring or outline that paints on every page landing or load, or an app still grey after a design port every instrument called green — the three ways a correct rule paints nothing (matches nothing / matches chrome / loses the cascade), the two reads that tell them apart, and the class that arrived in the stylesheet and is bound to no widget | | `skills/learned-detection-gaps.md` | Before trusting a green check/exec/DESCRIBE result as proof, or when a runtime symptom appears over a fully green model — the register of constructs that pass early rungs and fail later ones | diff --git a/agents/mdl-agent.md b/agents/mdl-agent.md index 33be1bb0..92850132 100644 --- a/agents/mdl-agent.md +++ b/agents/mdl-agent.md @@ -47,6 +47,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `project-bin/page-fidelity.js` | After drafting and again after exec'ing any page script — scores the page MDL (or `mxcli describe` output on stdin) against its wireframe: headings/actions/content/classes, weighted. The scored companion to check-page-shell's binary gate; 32% median measured without it, 90% first-draft with it. Every run is appended to the project's docs/PAGE-FIDELITY.tsv — first non-stub row per page = first-build score of record vs the ≥80% target (forward-reference stubs score with --stub, exempt) | | `bin/exec-approval.sh` | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/bug-lookup.sh` | A CE error (mx check or mxbuild: CE0066, CE0117…), a cited BUG-n, or behavior that looks like a known mxcli quirk rather than a modeling mistake — `bin/bug-lookup.sh CE0117` / `BUG-102` / "keyword" / the whole pasted error message (ranked top 3) prints the matching ledger entries, so the session reads one entry, not the whole ledger | | `bin/file-bugs.sh` | You hit, or worked around, a bug in mxcli or the toolkit, or log a FINDINGS.md entry about either — `bin/file-bugs.sh` (the `--add` line is in learned-detection-gaps.md) keeps docs/bugs.jsonl, redacts, shows duplicates, files with `--auto` when PROJECT.md says `Bug filing: auto`, else only a human files (`!`, `--file PBUG-NN`, y/N, denied to agents), or prints new-issue links (`--links`, `--json`) | diff --git a/agents/review-agent.md b/agents/review-agent.md index d7be6d65..f0490d28 100644 --- a/agents/review-agent.md +++ b/agents/review-agent.md @@ -47,6 +47,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/module-review.md` | Is the module done? Reviewing any module before calling it done — the ONE pass, the once-over: build, gate, prove, LOOK (is it logical, does it look right, does it match our design, over every page not just the tested ones), confirm with the denominator stated | | `skills/testing-shape.md` | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `skills/learned-css-that-never-applied.md` | A CSS/SCSS or style change that appears to have done nothing (a colour or class that won't show), a focus ring or outline that paints on every page landing or load, or an app still grey after a design port every instrument called green — the three ways a correct rule paints nothing (matches nothing / matches chrome / loses the cascade), the two reads that tell them apart, and the class that arrived in the stylesheet and is bound to no widget | | `skills/learned-detection-gaps.md` | Before trusting a green check/exec/DESCRIBE result as proof, or when a runtime symptom appears over a fully green model — the register of constructs that pass early rungs and fail later ones | diff --git a/agents/test-agent.md b/agents/test-agent.md index 14e3eee4..a7507d4e 100644 --- a/agents/test-agent.md +++ b/agents/test-agent.md @@ -39,6 +39,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/module-review.md` | Is the module done? Reviewing any module before calling it done — the ONE pass, the once-over: build, gate, prove, LOOK (is it logical, does it look right, does it match our design, over every page not just the tested ones), confirm with the denominator stated | | `skills/testing-shape.md` | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/status.sh` | The first command of every session, and any time someone asks "where are we" or "what next" — one screen: stage, done/overdue, the ONE next action, from the instruments, never from memory | | `skills/retesting-learned-rules.md` | Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | diff --git a/bin/gate-check.sh b/bin/gate-check.sh index 4c15089d..299cba71 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -2207,15 +2207,29 @@ stage4_closing_rows() { # own "Marketplace installs:" line and never changes this verdict, so a project whose Stage 5 # exit is already recorded cannot flip to "approved over a failing gate". It blocks earlier, in # build-ready and in exec.sh before a domain-model script, where the install belongs. +# +# A checkout without the ledger (#T3, two field runs 2026-11): .claude/loop/ is git-ignored, so a +# copy of the repo has no owed.tsv and used to stay MANUAL over a build with 0 of ~78 pages looked +# at. docs/progress/look-summary.tsv is the TRACKED summary look-ledger.sh writes (bin/ready.sh, +# close-task.sh); its `unseen` rows are judged exactly like OWED ledger rows below, so PROOF-OF-LOOK +# citations and waivers still clear them. MXTK_LOOK_MODULE= scopes the check to one module +# (bin/ready.sh --module); unset, nothing changes. check_stage_5() { local owed="$PROJECT_DIR/.claude/loop/look/owed.tsv" rows proofs="" n=0 n_seen=0 n_proof=0 n_waived=0 - local unseen="" n_unseen=0 st page epoch mod rep line shot f sz mt - if [ ! -s "$owed" ] || [ ! -f "$TOOLKIT_DIR/project-bin/look-ledger.sh" ]; then - check_stage_manual; return + local unseen="" n_unseen=0 st page epoch mod rep line shot f sz mt src="" + local summ="$PROJECT_DIR/docs/progress/look-summary.tsv" scope="${MXTK_LOOK_MODULE:-}" + if [ -s "$owed" ] && [ -f "$TOOLKIT_DIR/project-bin/look-ledger.sh" ]; then + # The toolkit's copy, not the project's: the same join for every project, whichever version + # of the script it installed. + rows="$(PROJECT_ROOT="$PROJECT_DIR" bash "$TOOLKIT_DIR/project-bin/look-ledger.sh" status 2>/dev/null)" + elif [ -s "$summ" ]; then + # SEEN rows are not listed by page in the summary: one placeholder per seen page of the scope. + rows="$(awk -F'\t' -v m="$scope" ' + $1=="module" && (m=="" || $2==m) { for (i=0; i<$4+0; i++) print "SEEN\t" $2 ".(seen)\t0" } + $1=="unseen" { print "OWED\t" $2 "\t" $3 }' "$summ")" + src=" (from the committed docs/progress/look-summary.tsv, $(awk -F'\t' '$1=="as-of"{print "as of " $2; exit}' "$summ"); the ledger is not in this checkout)" fi - # The toolkit's copy, not the project's: the same join for every project, whichever version - # of the script it installed. - rows="$(PROJECT_ROOT="$PROJECT_DIR" bash "$TOOLKIT_DIR/project-bin/look-ledger.sh" status 2>/dev/null)" + [ -n "$scope" ] && rows="$(printf '%s\n' "$rows" | awk -F'\t' -v m="$scope" 'index($2, m ".") == 1')" [ -n "$rows" ] || { check_stage_manual; return; } # PROOF-OF-LOOK citations, once: "Module.Pageshot-mtime" per citation with a real shot. @@ -2260,9 +2274,9 @@ EOF [ "$n_proof" -gt 0 ] && how="$how, $n_proof by PROOF-OF-LOOK" [ "$n_waived" -gt 0 ] && how="$how, $n_waived waived" if [ "$n_unseen" -gt 0 ]; then - echo "FAIL|look: $n_unseen of $n built page(s) never looked at since their last build: $unseen — screenshot each, open the PNG (a file name containing the page name, or the journey screenshot bin/look-ledger.sh shots lists), compare it to its wireframe (skills/ui-loop.md); or --waive look/ --reason \"...\"" + echo "FAIL|look: $n_unseen of $n built page(s)${scope:+ in $scope} never looked at since their last build$src: $unseen — screenshot each, open the PNG (a file name containing the page name, or the journey screenshot bin/look-ledger.sh shots lists), compare it to its wireframe (skills/ui-loop.md); or --waive look/ --reason \"...\"" else - echo "MANUAL|look: $n of $n built page(s) accounted for ($how); the rest of Stage 5 is manual" + echo "MANUAL|look: $n of $n built page(s)${scope:+ in $scope} accounted for ($how)$src; the rest of Stage 5 is manual" fi } diff --git a/bin/init-project.sh b/bin/init-project.sh index 70f9f89c..cbae653d 100755 --- a/bin/init-project.sh +++ b/bin/init-project.sh @@ -337,6 +337,8 @@ This project uses the shared toolkit at \`$TOOLKIT_ROOT\`. For ANY pipeline work 3. **Every gate question is asked in chat, then the turn ends and waits** (runbook §1). \`ASSUMED\` is earned by asking — the user said "you decide" — never by skipping. 4. **Before calling a stage done:** \`$TOOLKIT_ROOT/bin/gate-check.sh \`. + **READY / done on a module or the app only via** \`$TOOLKIT_ROOT/bin/ready.sh [--module M]\` + — it writes the READY line and commits the evidence; never write READY by hand. 5. **Before any page MDL:** read \`$TOOLKIT_ROOT/skills/ui-preflight-pages.md\` in full — its *text*; a delegated page draft carries the text or reads the file first. **After it:** the \`check-page-shell\`, \`page-fidelity\` and \`ui-loop\` rows below, then look at the page at diff --git a/bin/lib/skill-routing.tsv b/bin/lib/skill-routing.tsv index 7564a388..7b33dd3c 100644 --- a/bin/lib/skill-routing.tsv +++ b/bin/lib/skill-routing.tsv @@ -124,6 +124,7 @@ exec-approval bin/exec-approval.sh Before any mxcli exec / exec.sh / --mcp write context-pack project-bin/context-pack.sh Dispatching a build step to a helper — generate its one-file pack (the brief's Build steps row, mxcli brain brief, live DESCRIBEs, example, folder) and hand over the path instead of a reading list. Measured: 25-38% fewer tool calls and 30-41% less time, same quality; the token saving is not proven mdl 5 ondemand build testing-shape skills/testing-shape.md Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature test,gate,review 5,6 baseline verify verify-module project-bin/verify-module.sh Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh mdl,gate,test,review 5,6 baseline verify +ready bin/ready.sh Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand all 5,6 baseline verify security-check project-bin/security-check.sh Before hand-over, before calling a module security-ready, or after any change to roles, access rules or project security — security lint findings and the who-can-see-what snapshot, each compared with what a person accepted; exits 3 on a first run, never 0; a person runs --approve/--accept mdl,gate,test,review 5,6 ondemand build/security mxcli-bugs bug-logs/mxcli-bugs.md Reading a whole class of tool defects (a retest, a new mxcli release, an audit) — for one CE code or symptom use bin/bug-lookup.sh instead; the ledger is too large to read whole mdl,gate 5,6 ondemand diagnose bootstrap-project skills/bootstrap-project.md Generating a new project's CLAUDE.md — baseline routing plus project-specific facts ba P ondemand spine diff --git a/bin/ready.sh b/bin/ready.sh new file mode 100755 index 00000000..8c2b2240 --- /dev/null +++ b/bin/ready.sh @@ -0,0 +1,274 @@ +#!/usr/bin/env bash +# ready.sh — the ONE way a module or an app is called READY / done. It runs the checks, prints +# K of N for each, writes the READY line itself and commits the evidence behind it. +# +# Usage: bin/ready.sh [--module | --app ] [--no-commit] +# --module M scope every check to module M (its pages, its must-have list, its review) +# --app A the whole app, labelled A (default: the whole project, labelled "app") +# --no-commit write the files, leave committing to the caller +# Exit 0 READY, 1 NOT READY, 2 usage. Bash 3.2 compatible. +# +# WHY (two field runs, 2026-11, both unattended). Run 1: the build agent wrote READY into its own +# status file and was graded done with 0 of ~78 built pages ever looked at. The look ledger was +# installed and active, and gate-check.sh's Stage 5 check would have FAILed the claim — but +# nobody ran it before claiming, and .claude/loop/ is git-ignored, so the orchestrator and the +# reviewers looking at the repo saw nothing (gate-check on a copy said MANUAL). Every look that +# did happen was the builder judging its own screens. Run 2, another kit, marked itself ready +# 3 hours before its time cap with no styling or review pass. A READY anyone may write by hand +# is a claim; this script makes it a verdict, and puts the evidence in git. +# +# THE CHECKS (each row prints K of N; a row nobody measured is NOT MEASURED and fails — never +# green-by-absence): +# gate 5 gate-check.sh 5. PASS/WAIVED, or MANUAL with the look accounted for (Stage 5 +# is MANUAL by design once the look is clear — its other half is a Decisions row). +# gate 6 only when the register's "## Current stage" line is at Stage 6 or later. +# look pages owed a LOOK vs seen, per gate-check's own join (screenshots opened, +# PROOF-OF-LOOK citations, waivers — evidence this script did not create, guard +# rule 6). No page ever recorded as built and no `Waived obligation look` line: +# NOT MEASURED, a fail. +# must-have the per-module must-have list, when the project has one: K ticked of N. Absent: +# reported "not present", not counted. +# review only under `Interview mode: auto` (bin/interview-mode.sh): a verdict file written +# by a DIFFERENT agent (skills/module-review.md "Independent look"), every page of +# the scope scored 1-5 after its last build; a page below 4, or unscored, blocks. +# +# WHAT IT WRITES (all tracked, all small, no images): +# docs/progress/ready--.md the evidence: verdict, each row, owed vs seen per +# module, unseen pages, the screenshot names relied on +# docs/progress/ready.log one line per run: READY|NOT-READY +# — the READY line. An agent never writes it by hand. +# docs/progress/look-summary.tsv the git-ignored ledger's summary (look-ledger.sh summary) +# then `git commit -- ` (explicit paths: other sessions share the index). +# +# Producer for every consumer (CLAUDE.md "Shipping an instrument" rule 5): owed.tsv is written by +# exec.sh (the mandatory write path), seen.tsv by the Read hook install-claude-permissions.sh +# installs, the review file by the reviewer the lead dispatches (module-review.md), the must-have +# list by its own skill. ready.sh blocks nothing but the claim; it never blocks a write or a fix +# (guard rule 7): run it again after the fix. + +set -u +TOOLKIT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" + +usage() { echo "usage: ready.sh [--module | --app ] [--no-commit]" >&2; exit 2; } + +ROOT=""; MOD=""; APP=""; COMMIT=1 +while [ $# -gt 0 ]; do + case "$1" in + --module) shift; MOD="${1:-}"; [ -n "$MOD" ] || usage ;; + --app) shift; APP="${1:-}"; [ -n "$APP" ] || usage ;; + --no-commit) COMMIT=0 ;; + -h|--help) usage ;; + -*) echo "ready.sh: unknown option $1" >&2; usage ;; + *) [ -z "$ROOT" ] && ROOT="$1" || usage ;; + esac + shift +done +[ -n "$ROOT" ] && [ -d "$ROOT" ] || usage +[ -n "$MOD" ] && [ -n "$APP" ] && { echo "ready.sh: --module and --app are exclusive" >&2; usage; } +ROOT="$(cd "$ROOT" && pwd)" +REG="$ROOT/PROJECT.md" +if [ -n "$MOD" ]; then SCOPE="module $MOD"; elif [ -n "$APP" ]; then SCOPE="app $APP"; else SCOPE="app"; fi +SLUG="$(printf '%s' "${MOD:-${APP:-app}}" | tr -c 'A-Za-z0-9_-' '-' | sed 's/-*$//')" +DATE="$(date -u +%Y-%m-%d)"; ISO="$(date -u +%Y-%m-%dT%H:%M:%SZ)" +EVID_REL="docs/progress/ready-$DATE-$SLUG.md"; LOG_REL="docs/progress/ready.log" +LEDGER="$TOOLKIT_ROOT/project-bin/look-ledger.sh" +TMP="$(mktemp -d "${TMPDIR:-/tmp}/ready.XXXXXX")" || exit 2 +trap 'rm -rf "$TMP"' EXIT + +N_ROWS=0; N_OK=0 +: > "$TMP/rows.md" +# row — prints and records one check row +row() { + local mark + case "$1" in ok) mark="PASS"; N_OK=$((N_OK+1)); N_ROWS=$((N_ROWS+1)) ;; + fail) mark="FAIL"; N_ROWS=$((N_ROWS+1)) ;; + *) mark="n/a" ;; esac + printf ' %-4s %-9s %s\n' "$mark" "$2" "$3" + printf '| %s | %s | %s |\n' "$2" "$mark" "$(printf '%s' "$3" | sed 's/|/\\|/g')" >> "$TMP/rows.md" +} + +_epoch() { # ISO-8601 UTC -> epoch, GNU then BSD; 0 when unparseable + date -u -d "$1" +%s 2>/dev/null || date -u -j -f '%Y-%m-%dT%H:%M:%SZ' "$1" +%s 2>/dev/null || echo 0 +} + +echo "ready.sh — $SCOPE in $ROOT ($ISO)" + +# --- multi-app: a per-app scope needs the multi-app resolver; never guess it ------------------- +if [ -n "$APP" ] && ls "$ROOT"/apps/*/*.mpr >/dev/null 2>&1; then + row fail scope "NOT MEASURED — $ROOT holds several apps (apps/*/); per-app scoping of the look ledger is not resolved yet, so no check here can be read for app $APP alone" +fi + +# --- gate 5 --------------------------------------------------------------------------------- +GC="$TOOLKIT_ROOT/bin/gate-check.sh" +g5="$(MXTK_LOOK_MODULE="$MOD" bash "$GC" --no-html "$ROOT" 5 2>&1)"; rc5=$? +l5="$(printf '%s\n' "$g5" | grep -m1 '^Stage 5 (' | sed 's/^Stage 5 ([^)]*): //')" +v5="$(printf '%s' "$l5" | awk '{print $1}')" +look5="$(printf '%s' "$l5" | grep -o 'look: .*' | head -1)" +case "$rc5" in + 0) row ok gate5 "Stage 5 $v5" ;; + 2) if printf '%s' "$look5" | grep -q 'accounted for'; then row ok gate5 "Stage 5 MANUAL with the look accounted for (its sign-off is a Decisions row)" + else row fail gate5 "Stage 5 MANUAL with nothing measured — no page recorded as built"; fi ;; + *) row fail gate5 "Stage 5 ${v5:-exit $rc5}: $(printf '%s' "$l5" | sed 's/^[A-Z]* · //; s/^Surface MISSING: [^—]*— //' | cut -c1-200)" ;; +esac + +# --- gate 6, where the register's position calls for it -------------------------------------- +pos="" +[ -f "$REG" ] && pos="$(awk '/^## Current stage/{f=1;next} f && /^\*\*/{print;exit}' "$REG")" +case "$pos" in + *"Stage 6"*|*"Stage 7"*|*"All gates passed"*) + bash "$GC" --no-html "$ROOT" 6 > "$TMP/g6" 2>&1; rc6=$? + l6="$(grep -m1 '^Stage 6 (' "$TMP/g6" | sed 's/^Stage 6 ([^)]*): //')" + if [ "$rc6" = 0 ]; then row ok gate6 "Stage 6 $(printf '%s' "$l6" | awk '{print $1}') (register position: Stage 6 or later)" + else row fail gate6 "Stage 6 $(printf '%s' "$l6" | cut -c1-200)"; fi ;; + *) row info gate6 "not owed — the register's position is before Stage 6" ;; +esac + +# --- look: K of N, from gate-check's own join ------------------------------------------------- +waived_look() { + [ -f "$REG" ] || return 1 + grep -qiE "^[-*[:space:]]*Waived obligation look(/${MOD:-__none__})?[[:space:]]*:" "$REG" +} +if printf '%s' "$look5" | grep -q 'never looked at'; then + un="$(printf '%s' "$look5" | sed -n 's/^look: \([0-9]*\) of \([0-9]*\) .*/\1/p')" + tot="$(printf '%s' "$look5" | sed -n 's/^look: \([0-9]*\) of \([0-9]*\) .*/\2/p')" + row fail look "$((tot-un)) of $tot page(s) looked at since their last build — unseen: $(printf '%s' "$look5" | sed 's/^.*since their last build\( ([^)]*)\)\{0,1\}: //; s/ — screenshot.*//')" +elif printf '%s' "$look5" | grep -q 'accounted for'; then + tot="$(printf '%s' "$look5" | sed -n 's/^look: \([0-9]*\) of .*/\1/p')" + row ok look "$tot of $tot page(s) accounted for ($(printf '%s' "$look5" | sed -n 's/.*accounted for (\([^)]*\)).*/\1/p'))" +elif waived_look; then + row ok look "WAIVED — Waived obligation look${MOD:+/$MOD} in PROJECT.md" +else + row fail look "NOT MEASURED — no page${MOD:+ of $MOD} recorded as built (.claude/loop/look/owed.tsv, or the committed docs/progress/look-summary.tsv): the denominator is unknown. Build pages through exec.sh, or --waive look${MOD:+/$MOD} --reason \"...\" for a module with no pages" +fi + +# per-page facts for the evidence file and the review join: state, page, built-epoch, shot +if [ -s "$ROOT/.claude/loop/look/owed.tsv" ]; then + PROJECT_ROOT="$ROOT" bash "$LEDGER" summary --write >/dev/null 2>&1 + PROJECT_ROOT="$ROOT" bash "$LEDGER" status > "$TMP/st" 2>/dev/null + PROJECT_ROOT="$ROOT" bash "$LEDGER" status --shots > "$TMP/shots" 2>/dev/null +else + : > "$TMP/st"; : > "$TMP/shots" +fi +if [ -n "$MOD" ]; then + for f in st shots; do awk -F'\t' -v m="$MOD" 'index($2, m ".") == 1' "$TMP/$f" > "$TMP/$f.x"; mv "$TMP/$f.x" "$TMP/$f"; done +fi + +# --- must-have list (when the project has one) ----------------------------------------------- +MH="" +for f in ${MXTK_MUST_HAVES:-} ${MOD:+"$ROOT/architecture/modules/$MOD/must-haves.md"} "$ROOT/docs/must-haves.md"; do + [ -f "$f" ] && { MH="$f"; break; } +done +if [ -n "$MH" ]; then + # a module's own file counts whole; the shared file counts the lines naming the module (or all) + mh="$(awk -v m="$MOD" -v own="$( [ -n "$MOD" ] && [ "$MH" = "$ROOT/architecture/modules/$MOD/must-haves.md" ] && echo 1 || echo 0)" ' + /^[[:space:]]*[-*][[:space:]]+\[[ xX]\]/ { + if (m != "" && own != 1 && index($0, m) == 0) next + n++; if ($0 ~ /\[[xX]\]/) k++ + } END { print k+0 " " n+0 }' "$MH")" + k="${mh% *}"; n="${mh#* }" + if [ "$n" -eq 0 ]; then row fail must-have "0 of 0 — ${MH#"$ROOT"/} lists no checkbox item${MOD:+ for $MOD}: an empty list proves nothing" + elif [ "$k" -lt "$n" ]; then row fail must-have "$k of $n ticked in ${MH#"$ROOT"/}" + else row ok must-have "$n of $n ticked in ${MH#"$ROOT"/}"; fi +else + row info must-have "not present — no must-have list at ${MOD:+architecture/modules/$MOD/must-haves.md or }docs/must-haves.md" +fi + +# --- independent review, unattended mode only ------------------------------------------------- +MODE="$(bash "$TOOLKIT_ROOT/bin/interview-mode.sh" "$ROOT" 2>/dev/null)" +RV_DIR="$ROOT/docs/progress/look-reviews" +: > "$TMP/scores" +if [ "$MODE" = auto ]; then + for f in "$RV_DIR"/*.md; do + [ -f "$f" ] || continue + rvr="$(sed -n 's/^Reviewer:[[:space:]]*//p' "$f" | head -1 | tr -d '\r')" + bld="$(sed -n 's/^Builder:[[:space:]]*//p' "$f" | head -1 | tr -d '\r')" + at="$(sed -n 's/^Reviewed:[[:space:]]*//p' "$f" | head -1 | tr -d '\r' | awk '{print $1}')" + # a self-review is no review: both names present and different, or the file is not read + if [ -z "$rvr" ] || [ -z "$bld" ] || [ "$rvr" = "$bld" ]; then + echo " note review file ${f#"$ROOT"/} ignored: needs distinct Reviewer: and Builder: lines" >&2; continue + fi + ep="$(_epoch "$at")" + # table rows: | Module.Page | screenshot | target | score | note | — Score column found by header + tr -d '\r' < "$f" | awk -F'|' -v ep="$ep" -v fn="${f#"$ROOT"/}" ' + sc == 0 && /^\|/ { for (i=2; i 0 && /^\|/ { p=$2; gsub(/[[:space:]`]/, "", p); s=$sc; gsub(/[^0-9]/, "", s) + if (p ~ /^[A-Za-z0-9_]+\.[A-Za-z0-9_]+$/ && s != "") print p "\t" s "\t" ep "\t" fn }' >> "$TMP/scores" + done + if [ ! -s "$TMP/st" ]; then + row fail review "NOT MEASURED — no owed page to review${MOD:+ in $MOD}; the reviewer's denominator is the look ledger's pages" + else + # each owed page: the newest score given after its last build + # scores read in BEGIN, not as a first file: with an empty first file NR==FNR holds for the + # second one too, and the page list would be read as scores (0 of 0, "pass") + awk -F'\t' -v scf="$TMP/scores" ' + BEGIN { while ((getline line < scf) > 0) { split(line, f, "\t") + if (!(f[1] in at) || f[3]+0 > at[f[1]]) { at[f[1]]=f[3]+0; sc[f[1]]=f[2]+0 } } } + { p=$2; built=$3+0; n++ + if (!(p in at) || at[p] < built) { miss++; ml=ml (ml ? ", " : "") p; next } + if (sc[p] < 4) { low++; ll=ll (ll ? ", " : "") p "=" sc[p]; next } + k++ } + END { printf "%d|%d|%d|%d|%s|%s\n", k, n, miss, low, ml, ll }' "$TMP/st" > "$TMP/rv" + IFS='|' read -r k n miss low ml ll < "$TMP/rv" # not a tab: IFS whitespace collapses empty fields + if [ "$k" = "$n" ]; then row ok review "$n of $n page(s) scored 4+ by an independent reviewer (docs/progress/look-reviews/)" + else row fail review "$k of $n page(s) scored 4+ by an independent reviewer${ml:+ — unscored since last build: $ml}${ll:+ — below 4: $ll}. Dispatch a fresh reviewer (skills/module-review.md \"Independent look\")"; fi + fi +else + row info review "not owed — Interview mode is ${MODE:-steering}, not auto (a human sees the screens)" +fi + +# --- verdict, evidence, READY line ------------------------------------------------------------- +if [ "$N_OK" -eq "$N_ROWS" ] && [ "$N_ROWS" -gt 0 ]; then VERDICT=READY; else VERDICT=NOT-READY; fi +echo "$VERDICT — $N_OK of $N_ROWS checks pass ($SCOPE)" + +mkdir -p "$ROOT/docs/progress" +{ + echo "# ${VERDICT/-/ }: $SCOPE" + echo + echo "$VERDICT: $SCOPE — $ISO — $N_OK of $N_ROWS checks — written by bin/ready.sh, never by hand" + echo + echo "| Check | Verdict | Detail |" + echo "|---|---|---|" + cat "$TMP/rows.md" + echo + echo "## Pages owed a LOOK vs seen (look ledger)" + echo + if [ -s "$TMP/st" ]; then + echo "| Module | Owed | Seen |" + echo "|---|---|---|" + awk -F'\t' '{ m=$2; sub(/\..*/, "", m); if (!(m in o)) ord[++k]=m; o[m]++; if ($1=="SEEN") s[m]++ } + END { for (i=1; i<=k; i++) printf "| %s | %d | %d |\n", ord[i], o[ord[i]], s[ord[i]]+0 }' "$TMP/st" + echo + un="$(awk -F'\t' '$1=="OWED" { print $2 }' "$TMP/st")" + if [ -n "$un" ]; then + echo "Unseen since their last build ($(printf '%s\n' "$un" | wc -l | tr -d ' ')):" + printf '%s\n' "$un" | head -40 | sed 's/^/- /' + echo + fi + sh="$(awk -F'\t' '$1=="SEEN" && $3 != "" { b=$3; sub(/.*[\/\\]/, "", b); print "- " $2 ": " b }' "$TMP/shots")" + if [ -n "$sh" ]; then echo "Screenshots relied on:"; printf '%s\n' "$sh"; echo; fi + else + echo "No look ledger in this checkout (.claude/loop/look/ is git-ignored); see docs/progress/look-summary.tsv if committed." + echo + fi + if [ -n "$MH" ]; then echo "Must-have list: ${MH#"$ROOT"/}"; echo; fi + if [ -s "$TMP/scores" ]; then + echo "Review files read: $(cut -f4 "$TMP/scores" | sort -u | tr '\n' ' ')" + echo + fi +} > "$ROOT/$EVID_REL" +printf '%s\t%s\t%s\t%s\t%s of %s\n' "$ISO" "$VERDICT" "$SCOPE" "$EVID_REL" "$N_OK" "$N_ROWS" >> "$ROOT/$LOG_REL" +echo "evidence: $EVID_REL · log: $LOG_REL" + +# --- commit the evidence (explicit paths only) -------------------------------------------------- +if [ "$COMMIT" = 1 ] && git -C "$ROOT" rev-parse --git-dir >/dev/null 2>&1; then + paths="$EVID_REL $LOG_REL" + [ -f "$ROOT/docs/progress/look-summary.tsv" ] && paths="$paths docs/progress/look-summary.tsv" + # shellcheck disable=SC2086 + if ( cd "$ROOT" && git add -- $paths && git commit -q -m "ready: $SCOPE $VERDICT ($N_OK of $N_ROWS checks)" -- $paths ) >/dev/null 2>&1; then + echo "committed: $(git -C "$ROOT" log -1 --format='%h %s')" + else + echo " note the evidence was written but not committed (git refused) — commit these paths: $paths" >&2 + fi +fi + +[ "$VERDICT" = READY ] diff --git a/project-bin/close-task.sh b/project-bin/close-task.sh index 480a97c6..35050c75 100755 --- a/project-bin/close-task.sh +++ b/project-bin/close-task.sh @@ -119,6 +119,16 @@ if git rev-parse --git-dir >/dev/null 2>&1; then fi fi +# ---- the look ledger, as a tracked summary ------------------------------ +# .claude/loop/ is git-ignored, so without this "N owed, 0 seen" exists only in this container. +# look-ledger.sh rewrites docs/progress/look-summary.tsv (tracked) only when a count moved; the +# `git commit` below carries it. Fail-open: a missing ledger script prints nothing, owes nothing. +LOOK_SCRIPT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/look-ledger.sh" +if [ -f "$LOOK_SCRIPT" ] && [ -s "$ROOT/.claude/loop/look/owed.tsv" ]; then + look_line=$(PROJECT_ROOT="$ROOT" bash "$LOOK_SCRIPT" summary --write 2>/dev/null) + [ -n "$look_line" ] && { echo "### Look ledger"; echo; echo "- $look_line — commit docs/progress/look-summary.tsv with the rest"; echo "- READY / done is claimed only by \`$MXTK_ROOT/bin/ready.sh $ROOT [--module M]\`"; echo; } +fi + # ---- the routing table ------------------------------------------------- cat <<'TABLE' ### Where each thing goes — route by lifespan and audience, not topic diff --git a/project-bin/look-ledger.sh b/project-bin/look-ledger.sh index ef58751b..f0738619 100755 --- a/project-bin/look-ledger.sh +++ b/project-bin/look-ledger.sh @@ -3,11 +3,14 @@ # # bin/look-ledger.sh owe called by bin/exec.sh after a script lands # bin/look-ledger.sh seen PostToolUse(Read) hook; reads the hook JSON on stdin -# bin/look-ledger.sh status prints one line per owed page: SEEN or OWED +# bin/look-ledger.sh status [--shots] prints one line per owed page: SEEN or OWED (--shots: state, page, +# and the screenshot that saw it — bin/ready.sh's evidence list) # bin/look-ledger.sh reshoot [Module] [--all] # which pages a polish round must re-shoot (390 + 1280): the pages # built after their last seen LOOK, plus every page when a layout, # snippet or style file changed after it. K of N, never silence +# bin/look-ledger.sh summary [--write] pages owed vs opened, per module, plus each unseen page; --write +# also writes it to docs/progress/look-summary.tsv (TRACKED — see below) # bin/look-ledger.sh shots [findings.json...] # which owed page a journey run's screenshots already show: # OPEN (read it), UNREACHED (no journey did) @@ -361,13 +364,57 @@ RESHOOT_ROWS2 return 0 } +# cmd_summary [--write] — the ledger as a small TRACKED file, so "N owed, 0 seen" is visible from git. +# WHY (two field runs, 2026-11). .claude/loop/ is git-ignored, so the ledger lived only in the build's +# container: the orchestrator and the reviewers looking at the repo saw nothing, and gate-check.sh on a +# copy of the repo reported Stage 5 MANUAL for a build with 0 of ~78 pages ever looked at. The ledger +# itself stays ignored (it is append-only machine state, rewritten on every Read); this summary sits in +# docs/progress/ beside RESUME.md, which is tracked, instead of beside the ledger — a file inside an +# ignored directory cannot be re-included by a .gitignore negation. Producers: bin/ready.sh (every run) +# and close-task.sh (every work boundary, before its `git commit`). Consumer: gate-check.sh Stage 5, +# when the ledger is absent from the checkout. "seen" here is the raw ledger (a screenshot opened); +# gate-check also credits PROOF-OF-LOOK citations and waivers, so it reads the unseen rows, not the count. +# as-of owed seen unseen +# module +# unseen +SUMMARY_REL="docs/progress/look-summary.tsv" +_summary_body() { awk -F'\t' -v OFS='\t' '$1=="as-of" { $2="" } { print }'; } +cmd_summary() { + local rows out + rows="$(look_status)" + out="$(printf '%s\n' "$rows" | awk -F'\t' -v iso="$(date -u +%Y-%m-%dT%H:%M:%SZ)" ' + NF>=3 { m=$2; sub(/\..*/, "", m); if (!(m in o)) ord[++k]=m; o[m]++; n++ + if ($1=="SEEN") { s[m]++; ns++ } else un[++u]=$2 "\t" $3 } + END { + printf "# look-summary.tsv — written by look-ledger.sh summary from the git-ignored .claude/loop/look/ ledger; do not edit\n" + printf "as-of\t%s\towed %d\tseen %d\tunseen %d\n", iso, n, ns, n-ns + for (i=1; i<=k; i++) printf "module\t%s\t%d\t%d\n", ord[i], o[ord[i]], s[ord[i]]+0 + for (i=1; i<=u; i++) printf "unseen\t%s\n", un[i] + }')" + if [ "${1:-}" = --write ]; then + if [ -s "$OWED" ]; then + # rewritten only when a count or an unseen row moved: a new as-of alone is not worth a diff + if [ "$(_summary_body < "$PROJECT_ROOT/$SUMMARY_REL" 2>/dev/null)" != "$(printf '%s\n' "$out" | _summary_body)" ]; then + mkdir -p "$PROJECT_ROOT/docs/progress" 2>/dev/null && printf '%s\n' "$out" > "$PROJECT_ROOT/$SUMMARY_REL" 2>/dev/null + fi + fi + fi + if [ -s "$OWED" ]; then + printf '%s\n' "$out" | awk -F'\t' '$1=="as-of" { print "look ledger: " $3 ", " $4 ", " $5 " (as of " $2 ")" }' + else + echo "look ledger: NOT MEASURED — no $OWED (no page built through exec.sh in this checkout)" + fi + return 0 +} + case "${1:-}" in + summary) shift; cmd_summary "$@" ;; reshoot) shift; cmd_reshoot "$@" ;; owe) shift; cmd_owe "$@" ;; seen) cmd_seen ;; - status) look_status ;; + status) if [ "${2:-}" = --shots ]; then _look_status_ex | cut -f1,2,5; else look_status; fi ;; shots) shift; cmd_shots "$@" ;; pages) shift; look_pages_in "${1:-/dev/null}" ;; - *) echo "usage: $0 owe | seen < hook-json | status | reshoot [Module] [--all] | shots [findings.json...] | pages " >&2; exit 2 ;; + *) echo "usage: $0 owe | seen < hook-json | status [--shots] | summary [--write] | reshoot [Module] [--all] | shots [findings.json...] | pages " >&2; exit 2 ;; esac exit 0 diff --git a/skills/dispatching.md b/skills/dispatching.md index 50f77366..ba613141 100644 --- a/skills/dispatching.md +++ b/skills/dispatching.md @@ -104,6 +104,15 @@ every subagent: disable the ones a project does not use. The toolkit keeps its o (`CLAUDE.local.md`'s session-start ritual is main-session only and does not repeat what `gate-check.sh` prints). +**Independent look before READY (unattended).** Under `Interview mode: auto`, `bin/ready.sh` +refuses READY without `docs/progress/look-reviews/look-review-.md` from an agent that did +not build the pages. Dispatch a fresh reviewer, never the builder resumed. Its brief carries only +pairs: each page's screenshot and its target (wireframe, or the mapped prototype screen), never the +builder's verdict, notes or fidelity score. It writes `Reviewer:` and `Builder:` (different names), +`Reviewed: `, then a table `| Page | Screenshot | Target | Score | Note |`, one row +per page, Score 1–5 against the target. A page below 4, or rebuilt after its review, blocks READY. +Failure this prevents (field run, 2026-11): every look was the builder judging its own screens. + **Done when**, for each dispatch batch: N of N calls name a model; 0 scans started as forks; every fix round names its review file and commit range; 0 messages sent to agents that already reported. After a run, `bin/context-audit.sh --project ` section 1 gives the subagent start diff --git a/tests/wave2/test-ready.sh b/tests/wave2/test-ready.sh new file mode 100755 index 00000000..4b241d78 --- /dev/null +++ b/tests/wave2/test-ready.sh @@ -0,0 +1,144 @@ +#!/usr/bin/env bash +# test-ready.sh — bin/ready.sh: READY only when every check passes, K of N on every row, the +# evidence written and committed; plus the tracked look summary gate-check.sh reads from a copy. +# +# Golden input is captured, not imagined: the page names are the captured lab app's +# (fixtures/look-shots/show-pages-Approval.out, a finished Purchasing/Approval app, mxcli v0.28.0), +# the screenshots are that run's real PNGs, the owed ledger is written by the real +# `look-ledger.sh owe` from a build script in the captured statement shapes, and the seen ledger +# by the real `seen` hook from Claude Code's PostToolUse JSON. Nothing in .claude/loop/ is typed. +# +# Usage: bash tests/wave2/test-ready.sh [path-to-bin/ready.sh] + +SUBJECT="${1:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/bin/ready.sh}" +case "$SUBJECT" in /*) ;; *) SUBJECT="$PWD/$SUBJECT" ;; esac +TK="$(cd "$(dirname "$SUBJECT")/.." && pwd)" +FXS="$TK/tests/wave2/fixtures/look-shots" + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; } +WORK="$(mktemp -d "${TMPDIR:-/tmp}/readytest.XXXXXX")" || exit 2 +trap 'rm -rf "$WORK"' EXIT +echo "== subject: $SUBJECT" +[ -f "$SUBJECT" ] && [ -d "$FXS" ] || { echo "FAIL — subject or fixture missing"; echo "PASS=0 FAIL=1"; exit 1; } + +# ready.sh commits with the caller's git identity; a CI runner has none +export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@example.invalid GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@example.invalid +hook() { printf '{"hook_event_name":"PostToolUse","tool_name":"Read","tool_input":{"file_path":"%s"},"tool_response":{}}' "$1"; } +G() { git -C "$1" -c user.email=t@example.invalid -c user.name=t "${@:2}"; } + +# mkproject [mode] — a wired project: bin/ installed, three captured pages built +mkproject() { + mkdir -p "$1/bin" "$1/mdlsource" "$1/shots" "$1/docs/progress" + if [ "$2" = app ]; then mkdir -p "$1/app"; : > "$1/app/X.mpr"; else : > "$1/X.mpr"; fi + cp "$TK/project-bin/look-ledger.sh" "$TK/project-bin/_common.sh" "$1/bin/" + printf '# PROJECT.md\n\n## Current stage\n\n**Stage P — Kickoff**, in progress.\n\n%s\n' "${3:+Interview mode: $3}" > "$1/PROJECT.md" + printf '.claude/loop/\n' > "$1/.gitignore" + cat > "$1/mdlsource/20-approval-pages.mdl" <<'MDL' +CREATE PAGE "Approval"."Dashboard" +( + Title: 'Dashboard' +) { } +create or replace page Approval.Approval_Inbox (Title: 'Approval inbox') { } +CREATE OR MODIFY PAGE Approval.Approval_Decide { +} +MDL + (cd "$1" && bash bin/look-ledger.sh owe "$1/mdlsource/20-approval-pages.mdl" >/dev/null) + G "$1" init -q; G "$1" add -A; G "$1" commit -qm scaffold +} +# see — a screenshot named for its page, opened by the agent +see() { cp "$FXS/tests/e2e/artifacts/$3" "$1/shots/$2.png"; hook "$1/shots/$2.png" | (cd "$1" && bash bin/look-ledger.sh seen); } +see_all() { see "$1" dashboard-1280 J-DASH-01-step1.png; see "$1" approval_inbox-1280 J-APP-01-step1.png; see "$1" approval_decide-1280 J-APP-01-step2.png; } +# review +review() { + mkdir -p "$1/docs/progress/look-reviews" + cat > "$1/docs/progress/look-reviews/look-review-Approval.md" <&1)"; rc=$?; } + +# ── R1: owed but unseen pages → NOT READY, K of N, evidence committed ────────────────────── +P="$WORK/r1"; mkproject "$P" root +see "$P" dashboard-1280 J-DASH-01-step1.png +run "$P" +[ "$rc" = 1 ] && ok "R1: 1 of 3 seen → exit 1" || bad "R1: rc=$rc" +printf '%s\n' "$out" | grep -q 'FAIL look 1 of 3 page(s) looked at' && ok "R1: look row says 1 of 3" || bad "R1: $out" +printf '%s\n' "$out" | grep -q '^NOT-READY — 0 of 2 checks pass' && ok "R1: verdict NOT-READY with its denominator" || bad "R1: verdict: $out" +ev="$(ls "$P"/docs/progress/ready-*-app.md 2>/dev/null | head -1)" +[ -n "$ev" ] && grep -q '^- Approval.Approval_Inbox' "$ev" && grep -q 'Approval.Dashboard: dashboard-1280.png' "$ev" \ + && ok "R1: evidence names the unseen pages and the screenshot relied on" || bad "R1: evidence $(cat "$ev" 2>/dev/null)" +G "$P" ls-files --error-unmatch docs/progress/ready.log docs/progress/look-summary.tsv "${ev#"$P"/}" >/dev/null 2>&1 \ + && ok "R1: evidence, ready.log and look-summary.tsv are committed" || bad "R1: not committed: $(G "$P" status --short)" +grep -q '^as-of.*owed 3 seen 1 unseen 2' "$P/docs/progress/look-summary.tsv" && ok "R1: look-summary.tsv says owed 3 seen 1" || bad "R1: summary $(cat "$P/docs/progress/look-summary.tsv")" + +# ── R2: a copy of the repo (no ignored ledger) — gate-check reads the committed summary ──── +C="$WORK/r1-copy"; git clone -q "$P" "$C" 2>/dev/null +[ ! -e "$C/.claude/loop" ] || bad "R2: the clone has the ignored ledger" +g="$(bash "$TK/bin/gate-check.sh" --no-html "$C" 5 2>&1 | grep -m1 '^Stage 5 (')" +printf '%s' "$g" | grep -q 'FAIL.*2 of 3 built page(s) never looked at.*committed docs/progress/look-summary.tsv' \ + && ok "R2: a copy reports Stage 5 FAIL 2 of 3 from git, not MANUAL" || bad "R2: $g" +run "$C" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'look 1 of 3' && ok "R2: ready.sh on the copy is NOT READY, 1 of 3" || bad "R2: rc=$rc $out" + +# ── R3: all seen, steering → READY, the READY line written by the script ─────────────────── +see_all "$P" +run "$P" +[ "$rc" = 0 ] && ok "R3: all 3 seen → READY exit 0" || bad "R3: rc=$rc $out" +tail -1 "$P/docs/progress/ready.log" | grep -q ' READY app ' && ok "R3: ready.log carries the READY line" || bad "R3: log $(tail -1 "$P/docs/progress/ready.log")" +G "$P" log -1 --format=%s | grep -q '^ready: app READY (2 of 2 checks)' && ok "R3: committed as 'ready: app READY'" || bad "R3: $(G "$P" log -1 --format=%s)" +grep -q '^as-of.*owed 3 seen 3 unseen 0' "$P/docs/progress/look-summary.tsv" && ok "R3: summary refreshed to 3 of 3" || bad "R3: summary" + +# ── R4: unattended (auto) needs an independent review ────────────────────────────────────── +P="$WORK/r4"; mkproject "$P" root auto; see_all "$P" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'FAIL review 0 of 3 page(s) scored 4+' && ok "R4: auto, no reviewer file → NOT READY, review 0 of 3" || bad "R4: rc=$rc $out" +review "$P" other mdl-agent 3 +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'review 2 of 3 .*below 4: Approval.Approval_Decide=3' && ok "R4: a page scored 3 blocks, named" || bad "R4: $out" +review "$P" same mdl-agent 5 +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'ignored: needs distinct Reviewer: and Builder:' && ok "R4: a self-review (Reviewer = Builder) is not read" || bad "R4: $out" +review "$P" other mdl-agent 4 +run "$P" --no-commit +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q 'PASS review 3 of 3' && ok "R4: all seen + independent review 4+ → READY" || bad "R4: rc=$rc $out" +# a page rebuilt after the review is owed a new score (and a new look) +sleep 1; (cd "$P" && bash bin/look-ledger.sh owe "$P/mdlsource/20-approval-pages.mdl" >/dev/null); see_all "$P" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'review 0 of 3 .*unscored since last build' && ok "R4: a rebuild after the review voids its scores" || bad "R4: $out" + +# ── R5: --module scope; another module's unseen page does not block this one ─────────────── +P="$WORK/r5"; mkproject "$P" app; see_all "$P" +printf 'CREATE OR MODIFY PAGE Billing.Invoice_View {\n}\n' > "$P/mdlsource/30-billing.mdl" +(cd "$P" && bash bin/look-ledger.sh owe "$P/mdlsource/30-billing.mdl" >/dev/null) +run "$P" --module Approval --no-commit +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q 'look 3 of 3' && ok "R5 (two-tree): --module Approval READY, 3 of 3" || bad "R5: rc=$rc $out" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'look 3 of 4' && ok "R5 (two-tree): the whole app is NOT READY, 3 of 4" || bad "R5: rc=$rc $out" +run "$P" --module Nope --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'look NOT MEASURED' && ok "R5: a module with no built page is NOT MEASURED, never green" || bad "R5: $out" + +# ── R6: must-have list, when present ─────────────────────────────────────────────────────── +mkdir -p "$P/architecture/modules/Approval" +printf '# Must-haves — Approval\n- [x] inbox lists open requests\n- [ ] decide sends a notification\n' > "$P/architecture/modules/Approval/must-haves.md" +run "$P" --module Approval --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'must-have 1 of 2 ticked' && ok "R6: an unticked must-have blocks, 1 of 2" || bad "R6: $out" + +# ── R7: no ledger at all → NOT MEASURED ──────────────────────────────────────────────────── +P="$WORK/r7"; mkdir -p "$P"; printf '# PROJECT.md\n' > "$P/PROJECT.md" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'look NOT MEASURED' && ok "R7: no ledger → NOT READY, look NOT MEASURED" || bad "R7: rc=$rc $out" + +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] From c186bf7fa841d05ef0b426d041356a3af21a92f6 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 03:30:22 +0000 Subject: [PATCH 2/8] new(ci, ready): CI gate an agent cannot skip; READY nobody can hand-write bin/ci-gates.sh runs in the project's CI (templates/ci/toolkit-gates.yml, installed create-if-absent by init and sync) over the committed tree at the pinned toolkit commit. It fails on the claimed stage's gate FAIL, on a READY line bin/ready.sh did not write (bin/lib/ready-claims.sh, also a Stage 6 FAIL), on committed unseen pages under a done-claim, and on model work before Stage 4. It takes seconds and needs no Mendix runtime. ready.sh also writes a review packet (paths only, never images), reads T1's must-have mark on the tracked path docs/progress/must-haves/.md, and names every app of a multi-app project instead of checking nothing. Field evidence: the same two unattended field runs (2026-11). Exercised on fresh clones of projects built from the captured lab-app pages/screenshots. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y --- CHANGELOG.md | 1 + README.md | 2 +- ROUTING.md | 2 +- agents/architect-agent.md | 2 +- agents/ba-agent.md | 2 +- agents/gate-agent.md | 2 +- agents/mdl-agent.md | 2 +- agents/review-agent.md | 2 +- agents/test-agent.md | 2 +- bin/ci-gates.sh | 133 +++++++++++++++++++++++++++ bin/gate-check.sh | 11 ++- bin/init-project.sh | 5 ++ bin/lib/ci-template.sh | 28 ++++++ bin/lib/ready-claims.sh | 71 +++++++++++++++ bin/lib/resume-template.sh | 3 + bin/lib/skill-routing.tsv | 2 +- bin/ready.sh | 159 +++++++++++++++++++++++++-------- bin/sync-project.sh | 8 ++ templates/ci/toolkit-gates.yml | 45 ++++++++++ tests/wave2/test-ci-gates.sh | 128 ++++++++++++++++++++++++++ tests/wave2/test-ready.sh | 26 ++++-- 21 files changed, 580 insertions(+), 56 deletions(-) create mode 100755 bin/ci-gates.sh create mode 100644 bin/lib/ci-template.sh create mode 100644 bin/lib/ready-claims.sh create mode 100644 templates/ci/toolkit-gates.yml create mode 100755 tests/wave2/test-ci-gates.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index de9f1e02..c0e89e35 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m `docs/RELEASE-PROCESS.md` §8. ## Unreleased +- new(ci, ready): **the gate an agent cannot skip, and a READY nobody can hand-write.** `bin/ci-gates.sh` runs in the project's own CI (`.github/workflows/toolkit-gates.yml`, from `templates/ci/`, installed by `init-project.sh` and `sync-project.sh` create-if-absent, never overwritten, beside any existing CI) on every push and PR, over the committed tree at the pinned toolkit commit: the claimed stage's gate FAILs, a READY line `bin/ready.sh` did not write (`bin/lib/ready-claims.sh`, also a Stage 6 FAIL), committed unseen pages under a done-claim, and model work before Stage 4 all fail it, in seconds, with no Mendix runtime. `ready.sh` now writes a review packet (`docs/progress/review-packets/.html`: screenshot beside target with the reviewer's score, paths only), reads the must-have mark on its tracked path `docs/progress/must-haves/.md`, and names every app of a multi-app project instead of checking nothing (field runs, 2026-11: agents skip in-session gates; a hand-written READY graded a 0-of-78-looked build done) — two unattended field runs - new(ready, look-ledger, gate-check): **READY is earned through one command, and its evidence is in git.** `bin/ready.sh [--module M | --app A]` runs `gate-check.sh` Stage 5 (Stage 6 once the register is there), the look ledger, the must-have list when present and, under `Interview mode: auto`, an independent reviewer's verdict file (`docs/progress/look-reviews/`, every page scored 1–5 after its last build, below 4 blocks); it prints K of N per check, exits non-zero unless all pass, writes the READY line to `docs/progress/ready.log` itself and commits `docs/progress/ready--.md` with explicit paths. `look-ledger.sh summary --write` puts the git-ignored ledger into a tracked `docs/progress/look-summary.tsv` (written by `ready.sh` and every `close-task.sh`), and `gate-check.sh` Stage 5 reads it when the ledger is absent, so a copy of the repo reports FAIL instead of MANUAL. `dispatching.md` gains the reviewer dispatch rule; the `ready` routing row reaches `CLAUDE.local.md` and every agent stub (field runs, 2026-11: a build wrote READY by hand with 0 of ~78 pages looked at; another marked itself ready 3 hours early with no review pass) — two unattended field runs - fix(status, agents): **`bin/status.sh` WATCHes an Agent Editor project whose extension module does not travel with the model (#778)**: Studio Pro reads `agenteditor.agent` from `modules/AgentEditor.mxmodule` beside the `.mpr`, and a sync of only `.mpr` + `mprcontents/` left it out of git, so the builder stayed green while every colleague's clone showed `unknown type 'agenteditor.agent'`. Two WATCH items, warn-only (exit code and every verdict unchanged), local files and `git ls-files` only, root and `app/` layouts: a `modules/*.mxmodule` git does not track, and a project using Agent Editor (`AgentEditorCommons` in `analysis/app-facts/inventory.json`, or a `create agent` statement in `mdlsource/`) with no `modules/AgentEditor*.mxmodule`. `skills/mendix-agents.md` (on demand) gains the rule plus the note that the GenAI key, `Encryption.EncryptionKey` and `GenAICommons.StoreTraces` belong in the Default configuration, not a start script. Empty agent references / missing model version (mendixlabs/mxcli#1867) and unset runtime constants are not checked: they need a captured agent document (option 2). Fixture `tests/wave2/test-status-extension-modules.sh` — field report from an Agent Editor project, Nov 2026 - fix(permissions): **exec approval `auto` no longer prompts on every exec's mode check, and `install-claude-hooks.sh` no longer runs unprompted (#782)**: `install-claude-permissions.sh` now allow-lists `exec-approval.sh`, `interview-mode.sh` and `permission-profile.sh`, so their reads run without a prompt, while their only writers, `--set` and `--clear`, prompt through six `ask` rules (`Bash(*exec-approval.sh*--set*)`, …) in the shared settings under every profile (an ask rule beats an allow rule, `full`'s bare `Bash` included). `install-claude-hooks.sh`, which writes `~/.claude/settings.json` for every repo on the machine, gets no allow rule. A re-install adds the ask rules and removes the hooks-installer rule an earlier install added (sidecar); entries someone else wrote are untouched, and `sync-project.sh`'s permission check names both until then. Field run on claude 2.1.325: the read ran, `--set auto` and `--clear` were refused, and with the ask rules removed `--set` ran — from the two independent reviews of #781 diff --git a/README.md b/README.md index fa9e6378..6a97578b 100644 --- a/README.md +++ b/README.md @@ -914,7 +914,7 @@ Then filter by Role(s). A helper dispatched with a role stub (ba, architect, mdl | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | `bin/exec-approval.sh` | 5,6 | mdl, gate | | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | `skills/testing-shape.md` | 5,6 | test, gate, review | | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | `project-bin/verify-module.sh` | 5,6 | mdl, gate, test, review | -| Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | `bin/ready.sh` | 5,6 | every role | +| Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | `bin/ready.sh` | 5,6 | every role | | A CSS/SCSS or style change that appears to have done nothing (a colour or class that won't show), a focus ring or outline that paints on every page landing or load, or an app still grey after a design port every instrument called green — the three ways a correct rule paints nothing (matches nothing / matches chrome / loses the cascade), the two reads that tell them apart, and the class that arrived in the stylesheet and is bound to no widget | `skills/learned-css-that-never-applied.md` | 5,6 | review, gate (mdl: on demand) | | Before trusting a green check/exec/DESCRIBE result as proof, or when a runtime symptom appears over a fully green model — the register of constructs that pass early rungs and fail later ones | `skills/learned-detection-gaps.md` | 5,6 | gate, review (mdl: on demand) | | Creating any entity, or calling a module security-ready — entity and grants land in one script, and ready means SHOW SECURITY MATRIX proves it | `skills/security-is-not-a-later-script.md` | 5 | gate, review (mdl: on demand) | diff --git a/ROUTING.md b/ROUTING.md index 8c256c72..8f51fb5d 100644 --- a/ROUTING.md +++ b/ROUTING.md @@ -225,7 +225,7 @@ picks the row up. That is the whole procedure — there is no second list to rem |---|---|---|---|---| | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | `skills/testing-shape.md` | test,gate,review | 5,6 | baseline | | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | `project-bin/verify-module.sh` | mdl,gate,test,review | 5,6 | baseline | -| Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | `bin/ready.sh` | all | 5,6 | baseline | +| Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | `bin/ready.sh` | all | 5,6 | baseline | | Auditing or regression/e2e-testing an EXISTING app — no intake, no stages, no gates | `skills/existing-app-assurance.md` | test,review | 6 | ondemand | | Running the ledger rung alone — recompute every stored ledger status against the live model and catch the STALE rows that claim built for something the model no longer has | `project-bin/conformance-check.sh` | gate,review | 5,6 | ondemand | | Running the wiring rung alone — a module imported but never reached, an element built but wired to nothing, a boundary crossed; mxbuild and e2e are blind to all three | `project-bin/graph-sweep.sh` | review,gate | 5,6 | ondemand | diff --git a/agents/architect-agent.md b/agents/architect-agent.md index 58168c09..2b6a66ed 100644 --- a/agents/architect-agent.md +++ b/agents/architect-agent.md @@ -41,7 +41,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/checkpoints/checkpoint-template.md` | Any stage transition — the 2+1 format every CAC uses, and the one-register rule (answers land in PROJECT.md, never in a separate state file). The seven CACs themselves are routed per stage in the situational table | | `skills/module-folder-convention.md` | Placing any document in a module — before the first `create`. Feature group, then Pages/Microflows/Services/Resources; the path comes from the brief's folder plan, and the table says which types mxcli can actually place | | `project-bin/check-design-reaches-app.sh` | After the FIRST build that follows any design-system port, and before any page is built on it — reads the BUILT stylesheet and reports how many framework knobs point at a design token, how many tokens arrived, how many component classes arrived, each with its denominator. Measured on a real run: 55 tokens ported correctly into the right file, 0 of 35 knobs bound and 0 of 20 classes present, two build phases shipped in the framework's default blue with mx check, mxcli lint, the MDL suite and two e2e journeys all green | -| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/status.sh` | The first command of every session, and any time someone asks "where are we" or "what next" — one screen: stage, done/overdue, the ONE next action, from the instruments, never from memory | | `skills/retesting-learned-rules.md` | Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | diff --git a/agents/ba-agent.md b/agents/ba-agent.md index 1c14f9e4..7f3347bb 100644 --- a/agents/ba-agent.md +++ b/agents/ba-agent.md @@ -53,7 +53,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `bin/question-kinds.sh` | Deciding who answers a question — before putting any batch to the user. gap/conflict/choice/user-only is what keeps a gate batch at four questions instead of 127 | | `bin/facts-lock.sh` | Writing BRDs, especially several in parallel — "build" before the fan-out, "check" before any BRD is called done | | `skills/module-brief.md` | Building any module — before the first script. The mdl-agent's single per-module input | -| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/status.sh` | The first command of every session, and any time someone asks "where are we" or "what next" — one screen: stage, done/overdue, the ONE next action, from the instruments, never from memory | | `skills/retesting-learned-rules.md` | Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | diff --git a/agents/gate-agent.md b/agents/gate-agent.md index f1608ee8..f222b096 100644 --- a/agents/gate-agent.md +++ b/agents/gate-agent.md @@ -40,7 +40,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `bin/exec-approval.sh` | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | | `skills/testing-shape.md` | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | -| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `skills/learned-css-that-never-applied.md` | A CSS/SCSS or style change that appears to have done nothing (a colour or class that won't show), a focus ring or outline that paints on every page landing or load, or an app still grey after a design port every instrument called green — the three ways a correct rule paints nothing (matches nothing / matches chrome / loses the cascade), the two reads that tell them apart, and the class that arrived in the stylesheet and is bound to no widget | | `skills/learned-detection-gaps.md` | Before trusting a green check/exec/DESCRIBE result as proof, or when a runtime symptom appears over a fully green model — the register of constructs that pass early rungs and fail later ones | diff --git a/agents/mdl-agent.md b/agents/mdl-agent.md index 92850132..a36e1bfb 100644 --- a/agents/mdl-agent.md +++ b/agents/mdl-agent.md @@ -47,7 +47,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `project-bin/page-fidelity.js` | After drafting and again after exec'ing any page script — scores the page MDL (or `mxcli describe` output on stdin) against its wireframe: headings/actions/content/classes, weighted. The scored companion to check-page-shell's binary gate; 32% median measured without it, 90% first-draft with it. Every run is appended to the project's docs/PAGE-FIDELITY.tsv — first non-stub row per page = first-build score of record vs the ≥80% target (forward-reference stubs score with --stub, exempt) | | `bin/exec-approval.sh` | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | -| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/bug-lookup.sh` | A CE error (mx check or mxbuild: CE0066, CE0117…), a cited BUG-n, or behavior that looks like a known mxcli quirk rather than a modeling mistake — `bin/bug-lookup.sh CE0117` / `BUG-102` / "keyword" / the whole pasted error message (ranked top 3) prints the matching ledger entries, so the session reads one entry, not the whole ledger | | `bin/file-bugs.sh` | You hit, or worked around, a bug in mxcli or the toolkit, or log a FINDINGS.md entry about either — `bin/file-bugs.sh` (the `--add` line is in learned-detection-gaps.md) keeps docs/bugs.jsonl, redacts, shows duplicates, files with `--auto` when PROJECT.md says `Bug filing: auto`, else only a human files (`!`, `--file PBUG-NN`, y/N, denied to agents), or prints new-issue links (`--links`, `--json`) | diff --git a/agents/review-agent.md b/agents/review-agent.md index f0490d28..0d2a1b12 100644 --- a/agents/review-agent.md +++ b/agents/review-agent.md @@ -47,7 +47,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/module-review.md` | Is the module done? Reviewing any module before calling it done — the ONE pass, the once-over: build, gate, prove, LOOK (is it logical, does it look right, does it match our design, over every page not just the tested ones), confirm with the denominator stated | | `skills/testing-shape.md` | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | -| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `skills/learned-css-that-never-applied.md` | A CSS/SCSS or style change that appears to have done nothing (a colour or class that won't show), a focus ring or outline that paints on every page landing or load, or an app still grey after a design port every instrument called green — the three ways a correct rule paints nothing (matches nothing / matches chrome / loses the cascade), the two reads that tell them apart, and the class that arrived in the stylesheet and is bound to no widget | | `skills/learned-detection-gaps.md` | Before trusting a green check/exec/DESCRIBE result as proof, or when a runtime symptom appears over a fully green model — the register of constructs that pass early rungs and fail later ones | diff --git a/agents/test-agent.md b/agents/test-agent.md index a7507d4e..330dafaf 100644 --- a/agents/test-agent.md +++ b/agents/test-agent.md @@ -39,7 +39,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/module-review.md` | Is the module done? Reviewing any module before calling it done — the ONE pass, the once-over: build, gate, prove, LOOK (is it logical, does it look right, does it match our design, over every page not just the tested ones), confirm with the denominator stated | | `skills/testing-shape.md` | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | -| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/status.sh` | The first command of every session, and any time someone asks "where are we" or "what next" — one screen: stage, done/overdue, the ONE next action, from the instruments, never from memory | | `skills/retesting-learned-rules.md` | Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | diff --git a/bin/ci-gates.sh b/bin/ci-gates.sh new file mode 100755 index 00000000..ca42d004 --- /dev/null +++ b/bin/ci-gates.sh @@ -0,0 +1,133 @@ +#!/usr/bin/env bash +# ci-gates.sh — the gate an agent cannot skip. Run by the project's CI (.github/workflows/toolkit-gates.yml, +# installed by init-project.sh / sync-project.sh from templates/ci/toolkit-gates.yml) on every push and PR, +# over the COMMITTED tree only, with the toolkit checked out at the commit PROJECT.md pins. +# +# Usage: bin/ci-gates.sh exit 0 clean (warnings allowed), 1 a gate failed, 2 usage +# MXTK_CI_STRICT_LOOK=1 unseen built pages fail even with no done-claim (default: warn mid-build) +# +# WHY (field runs, 2026-11, unattended). Everything that runs inside the agent's session can be skipped +# by the agent: it can not run gate-check, commit with --no-verify, or write READY by hand — and in one +# run it did the last, with 0 of ~78 pages looked at. CI runs outside the session, on what was pushed. +# +# THE CHECKS +# (a) stage gate-check.sh for the stage the register's "## Current stage" line claims; FAIL fails. +# PENDING / MANUAL do not (a stage in progress has nothing to show yet). +# (b) ready a READY line in a status file that bin/ready.sh did not write (bin/lib/ready-claims.sh). +# (c) look the committed docs/progress/look-summary.tsv shows built pages never looked at, while +# something claims done: a READY row in docs/progress/ready.log covering them, or the +# register at Stage 6 or later. With no claim it is a warning (mid-build is normal and an +# always-red CI gets ignored); MXTK_CI_STRICT_LOOK=1 makes it fail. A READY with no +# committed summary behind it fails: the claim cites evidence nobody can read. +# (d) order model work before its plan: docs/BUILD-LOG.md has exec rows (or mdlsource/ holds build +# scripts outside gallery/) while gate-check Stage 4 is not PASS/WAIVED. gate-check already +# waives Stage 4 for greenfield and for an --adopt point at or after it. +# +# WHAT IT CANNOT SEE (anything not committed): the git-ignored .claude/loop/ ledgers themselves (only +# the summary ready.sh / close-task.sh commit), screenshots, chat, a model edited without exec.sh and +# without a BUILD-LOG row, and a summary nobody committed. It reads files, never the .mpr: no Mendix +# runtime, no mxcli. It runs in seconds (three gate-check stage queries at most). +# +# Producer for every consumer (CLAUDE.md "Shipping an instrument" rule 5): look-summary.tsv and ready.log +# are written by bin/ready.sh and close-task.sh; BUILD-LOG rows by exec.sh; the register by the gates. + +set -u +TOOLKIT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +ROOT="${1:-}" +[ -n "$ROOT" ] && [ -d "$ROOT" ] || { echo "usage: ci-gates.sh " >&2; exit 2; } +ROOT="$(cd "$ROOT" && pwd)" +REG="$ROOT/PROJECT.md"; GC="$TOOLKIT_ROOT/bin/gate-check.sh" +SUMM="$ROOT/docs/progress/look-summary.tsv"; LOG="$ROOT/docs/progress/ready.log" +FAILS=0; WARNS=0 +say() { # + case "$1" in FAIL) FAILS=$((FAILS+1)) ;; WARN) WARNS=$((WARNS+1)) ;; esac + printf '%-4s %-6s %s\n' "$1" "$2" "$3" + if [ -n "${GITHUB_ACTIONS:-}" ]; then + case "$1" in FAIL) echo "::error title=toolkit gate $2::$3" ;; WARN) echo "::warning title=toolkit gate $2::$3" ;; esac + fi +} +# gate — sets GV (verdict word) and GL (detail), returns gate-check's exit code +gate() { + local out rc + out="$(MXTK_NO_GUIDE=1 bash "$GC" --no-html "$ROOT" "$1" 2>&1)"; rc=$? + GL="$(printf '%s\n' "$out" | grep -m1 "^Stage $1 (" | sed "s/^Stage $1 ([^)]*): //")" + GV="$(printf '%s' "$GL" | awk '{print $1}')" + return $rc +} + +echo "ci-gates — $ROOT (toolkit $(git -C "$TOOLKIT_ROOT" rev-parse --short HEAD 2>/dev/null || echo '?'))" +[ -f "$REG" ] || { say FAIL stage "no PROJECT.md: not a toolkit project, or the register was never committed"; echo "FAIL — 1 gate"; exit 1; } + +# (a) the stage the register claims +pos="$(awk '/^## Current stage/{f=1;next} f && /^\*\*/{print;exit}' "$REG")" +case "$pos" in + *"All gates passed"*) claim=7 ;; + *) claim="$(printf '%s' "$pos" | grep -oE 'Stage [0-9P]' | head -1 | sed 's/^Stage //')" ;; +esac +if [ -z "$claim" ]; then + say WARN stage "no \"## Current stage\" line with a stage in PROJECT.md — nothing claimed, nothing to check" +else + gate "$claim"; rc=$? + case "$rc" in + 1) say FAIL stage "the register claims Stage $claim and its gate FAILs: $(printf '%s' "$GL" | cut -c1-240)" ;; + *) say PASS stage "Stage $claim: ${GV:-exit $rc}" ;; + esac +fi + +# (b) READY written by hand +. "$TOOLKIT_ROOT/bin/lib/ready-claims.sh" +if rcf="$(mxtk_ready_claim_faults "$ROOT")"; then + say PASS ready "every READY line is backed by bin/ready.sh evidence" +else + while IFS= read -r l; do [ -n "$l" ] && say FAIL ready "$l"; done <= 4 { last[$3] = $2 } END { for (s in last) if (last[s] == "READY") print s }' "$LOG")" +late=0; case "$claim" in 6|7) late=1 ;; esac +if [ -s "$SUMM" ]; then + unseen="$(awk -F'\t' '$1 == "unseen" { print $2 }' "$SUMM")" + asof="$(awk -F'\t' '$1 == "as-of" { print $2 " (" $3 ", " $4 ")"; exit }' "$SUMM")" + if [ -z "$unseen" ]; then + say PASS look "committed look summary: no built page unseen, as of $asof" + else + n="$(printf '%s\n' "$unseen" | wc -l | tr -d ' ')" + hit="" + while IFS= read -r sc; do + [ -n "$sc" ] || continue + case "$sc" in + "module "*) m="${sc#module }"; printf '%s\n' "$unseen" | grep -q "^$m\." && hit="$hit${hit:+; }$sc" ;; + *) hit="$hit${hit:+; }$sc" ;; + esac + done </dev/null)" +scripts="$( [ -d "$ROOT/mdlsource" ] && find "$ROOT/mdlsource" -name '*.mdl' ! -path '*/gallery/*' 2>/dev/null | wc -l | tr -d ' ')" +if [ "${execs:-0}" -gt 0 ] || [ "${scripts:-0}" -gt 0 ]; then + gate 4; rc=$? + if [ "$rc" = 0 ]; then say PASS order "model work ($execs exec row(s), $scripts script(s)) after Stage 4: $GV" + else say FAIL order "model work ($execs exec row(s) in docs/BUILD-LOG.md, $scripts build script(s) in mdlsource/) while Stage 4 is ${GV:-not passed}: build before the plan. Pass Stage 4, or record the adoption point (gate-check.sh --adopt 5 --reason \"...\")"; fi +else + say PASS order "no model work committed yet" +fi + +if [ "$FAILS" -gt 0 ]; then echo "FAIL — $FAILS gate(s) failed, $WARNS warning(s)"; exit 1; fi +echo "PASS — $WARNS warning(s)" +exit 0 diff --git a/bin/gate-check.sh b/bin/gate-check.sh index 299cba71..582da7e2 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -2298,7 +2298,16 @@ test_surface_hit() { } check_stage_6() { - local test_ok="" review_ok="" + local test_ok="" review_ok="" rcf="" + # A READY line bin/ready.sh did not write is not READY (bin/lib/ready-claims.sh): Stage 6 FAILs on + # it before anything else, so a hand-written READY cannot carry a project into its test stage. + if [ -r "$TOOLKIT_DIR/bin/lib/ready-claims.sh" ]; then + . "$TOOLKIT_DIR/bin/lib/ready-claims.sh" + if ! rcf="$(mxtk_ready_claim_faults "$PROJECT_DIR")"; then + echo "FAIL|READY claimed without bin/ready.sh ($(printf '%s\n' "$rcf" | grep -c .) line(s)): $(printf '%s\n' "$rcf" | head -3 | tr '\n' ';' | sed 's/;$//') — READY is written only by bin/ready.sh [--module M]; remove the hand-written line or run it" + return + fi + fi test_surface_hit && test_ok=1 # module-review.md report — any non-empty dated report under a ui-reviews/ dir if [ -n "$(find_artifact -path '*/ui-reviews/ui-review-*.html' -size +0c)" ]; then diff --git a/bin/init-project.sh b/bin/init-project.sh index cbae653d..28af150d 100755 --- a/bin/init-project.sh +++ b/bin/init-project.sh @@ -290,6 +290,11 @@ else echo "Created: docs/progress/RESUME.md (starter — the first close-the-loop overwrites it)" fi +# .github/workflows/toolkit-gates.yml — the CI gate an agent cannot skip (bin/ci-gates.sh). Create-if- +# absent, never overwritten (bin/lib/ci-template.sh). +. "$TOOLKIT_ROOT/bin/lib/ci-template.sh" +mxtk_ci_install "$TOOLKIT_ROOT" "$PROJECT_DIR" 0 || true + CLAUDE_LOCAL="$PROJECT_DIR/CLAUDE.local.md" if [ -f "$CLAUDE_LOCAL" ]; then echo "Skip: CLAUDE.local.md already exists — not overwritten." diff --git a/bin/lib/ci-template.sh b/bin/lib/ci-template.sh new file mode 100644 index 00000000..e3d3dcc0 --- /dev/null +++ b/bin/lib/ci-template.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash +# ci-template.sh — install the project's CI gate (templates/ci/toolkit-gates.yml) as +# .github/workflows/toolkit-gates.yml. ONE copy, sourced by bin/init-project.sh and bin/sync-project.sh, +# for the same reason as ritual-template.sh: two hand-kept installers drift. +# +# Contract: CREATE-IF-ABSENT, never overwrite. The file is the project's once it exists (it may have +# been adapted: a fork's repo URL, a token, a runner); a copy that differs from the template is only +# reported. Other workflows in .github/workflows/ are left untouched: this is a separate workflow, +# not an edit of an existing one, so it sits beside whatever CI the repo already has. +# +# Sourced, never executed. Side-effect free until called. + +# mxtk_ci_install [dry-run 0|1] — prints one line; returns 0 when it +# created the file (a change), 1 otherwise. +mxtk_ci_install() { + local tk="$1" pr="$2" dry="${3:-0}" src dst other + src="$tk/templates/ci/toolkit-gates.yml"; dst="$pr/.github/workflows/toolkit-gates.yml" + [ -f "$src" ] || return 1 + if [ -f "$dst" ]; then + cmp -s "$src" "$dst" || echo "Kept: .github/workflows/toolkit-gates.yml differs from the toolkit template (yours, not overwritten; compare with $src)" + return 1 + fi + other="$(ls "$pr/.github/workflows" 2>/dev/null | grep -E '\.ya?ml$' | tr '\n' ' ')" + if [ "$dry" = 1 ]; then echo "Would create: .github/workflows/toolkit-gates.yml (the CI gate an agent cannot skip)"; return 0; fi + mkdir -p "$pr/.github/workflows" && cp "$src" "$dst" || return 1 + echo "Created: .github/workflows/toolkit-gates.yml — the CI gate an agent cannot skip (bin/ci-gates.sh on every push and PR)${other:+; existing workflows left untouched: $other}" + return 0 +} diff --git a/bin/lib/ready-claims.sh b/bin/lib/ready-claims.sh new file mode 100644 index 00000000..91d3bafa --- /dev/null +++ b/bin/lib/ready-claims.sh @@ -0,0 +1,71 @@ +#!/usr/bin/env bash +# ready-claims.sh — a READY line that bin/ready.sh did not write is not READY. ONE copy, sourced by +# bin/gate-check.sh (Stage 6) and bin/ci-gates.sh (the CI gate an agent cannot skip). +# +# WHY (field run, 2026-11, unattended). The build agent wrote READY into its own status file and +# was graded done with 0 of ~78 pages looked at. bin/ready.sh is now the only writer of READY; this +# is the check that a READY line anywhere else is backed by it. +# +# A READY CLAIM is a line in a status file whose first word (after list, quote and bold markup, or +# a `Status:` / `State:` / `Verdict:` label) is READY, or a table cell that is exactly READY. +# Status files: PROJECT.md, RESUME.md / STATUS*.md / PROGRESS*.md at the root, and every .md under +# docs/ except ready.sh's own output (docs/progress/ready-*.md, review-packets/, look-reviews/, +# must-haves/). Not CLAUDE*.md, README or agent text: they explain the rule, they claim nothing. +# BACKED means the line names an evidence file docs/progress/ready--.md that exists, +# whose verdict line (line 3, written by ready.sh) starts "READY:", and which docs/progress/ready.log +# lists as READY. Each scope's latest row in ready.log is held to the same test, so a hand-appended +# READY row with no evidence behind it is caught too; an older READY that a later run withdrew is not. +# +# What it cannot see: a claim made only in chat, a commit message, or an uncommitted file. A +# hand-forged evidence file passes this check; bin/ci-gates.sh then compares it with the committed +# look summary, which is where a forged claim and the ledger disagree. +# +# Sourced, never executed. Side-effect free. Bash 3.2. + +# mxtk_ready_claim_faults — one line per unbacked READY claim: +# :: — +# Returns 0 when there is none, 1 when there is at least one. +mxtk_ready_claim_faults() { + local root="$1" f rel log="$1/docs/progress/ready.log" out="" ev line n text why + _rc_backed() { # — 0 when ready.sh's READY evidence and its log row agree + [ -f "$root/$1" ] || return 1 + sed -n 3p "$root/$1" | grep -q '^READY:' || return 1 + [ -f "$log" ] && awk -F'\t' -v e="$1" '$2 == "READY" && $4 == e { f=1 } END { exit !f }' "$log" + } + { + for f in "$root/PROJECT.md" "$root"/RESUME.md "$root"/STATUS*.md "$root"/PROGRESS*.md; do [ -f "$f" ] && printf '%s\n' "$f"; done + [ -d "$root/docs" ] && find "$root/docs" -type f -name '*.md' \ + ! -path '*/docs/progress/ready-*' ! -path '*/docs/progress/review-packets/*' \ + ! -path '*/docs/progress/look-reviews/*' ! -path '*/docs/progress/must-haves/*' 2>/dev/null + } | sort -u > "${TMPDIR:-/tmp}/mxtk-rc.$$" + while IFS= read -r f; do + rel="${f#"$root"/}" + while IFS= read -r line; do + [ -n "$line" ] || continue + n="${line%%:*}"; text="${line#*:}" + ev="$(printf '%s' "$text" | grep -oE 'docs/progress/ready-[A-Za-z0-9_.-]+\.md' | head -1)" + if [ -z "$ev" ]; then why="no bin/ready.sh evidence named (docs/progress/ready--.md)" + elif ! _rc_backed "$ev"; then why="$ev is not a READY written by bin/ready.sh (missing, not READY, or not in docs/progress/ready.log)" + else continue; fi + out="$out$rel:$n: $(printf '%s' "$text" | cut -c1-100) — $why +" + done <*_#-]*(\*\*)?((Status|State|Verdict)[[:space:]]*:[[:space:]]*)?(\*\*)?READY(\*\*)?([[:space:]]*$|[[:space:]]*[:(]|[[:space:]]+[—–-]|[[:space:]]+(for|—)[[:space:]])|\|[[:space:]]*(\*\*)?READY(\*\*)?[[:space:]]*\|') +EOF + done < "${TMPDIR:-/tmp}/mxtk-rc.$$" + rm -f "${TMPDIR:-/tmp}/mxtk-rc.$$" + if [ -f "$log" ]; then + # Only each scope's LATEST row is a standing claim: a later NOT-READY run withdraws an earlier + # READY (and overwrites that day's evidence file), so judging old rows would fault an honest log. + while IFS=' ' read -r _iso v sc ev _k; do + [ "$v" = READY ] || continue + _rc_backed "$ev" || out="${out}docs/progress/ready.log: READY $sc — $ev is missing or does not say READY +" + done <= 4 { last[$3] = $0; if (!($3 in o)) ord[++k] = $3; o[$3] = 1 } END { for (i=1; i<=k; i++) print last[ord[i]] }' "$log") +EOF + fi + [ -z "$out" ] && return 0 + printf '%s' "$out" + return 1 +} diff --git a/bin/lib/resume-template.sh b/bin/lib/resume-template.sh index 1a079f60..a202fc74 100644 --- a/bin/lib/resume-template.sh +++ b/bin/lib/resume-template.sh @@ -50,6 +50,9 @@ it). The first session to finish work overwrites it wholesale per confidently wrong on any project that was already mid-flight when this file appeared, and a wrong resume doc is worse than none. Step 1 below is how to find out. +**READY is not written in this file.** Only \`$toolkit/bin/ready.sh\` writes it +(\`docs/progress/ready.log\` plus its evidence); a READY line it did not write FAILs Stage 6 and CI. + ## Next actions, in order 1. Run \`$toolkit/bin/gate-check.sh .\` — it reports the current stage from the files on disk. diff --git a/bin/lib/skill-routing.tsv b/bin/lib/skill-routing.tsv index 7b33dd3c..ca953660 100644 --- a/bin/lib/skill-routing.tsv +++ b/bin/lib/skill-routing.tsv @@ -124,7 +124,7 @@ exec-approval bin/exec-approval.sh Before any mxcli exec / exec.sh / --mcp write context-pack project-bin/context-pack.sh Dispatching a build step to a helper — generate its one-file pack (the brief's Build steps row, mxcli brain brief, live DESCRIBEs, example, folder) and hand over the path instead of a reading list. Measured: 25-38% fewer tool calls and 30-41% less time, same quality; the token saving is not proven mdl 5 ondemand build testing-shape skills/testing-shape.md Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature test,gate,review 5,6 baseline verify verify-module project-bin/verify-module.sh Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh mdl,gate,test,review 5,6 baseline verify -ready bin/ready.sh Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; never write READY in a status file by hand all 5,6 baseline verify +ready bin/ready.sh Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI all 5,6 baseline verify security-check project-bin/security-check.sh Before hand-over, before calling a module security-ready, or after any change to roles, access rules or project security — security lint findings and the who-can-see-what snapshot, each compared with what a person accepted; exits 3 on a first run, never 0; a person runs --approve/--accept mdl,gate,test,review 5,6 ondemand build/security mxcli-bugs bug-logs/mxcli-bugs.md Reading a whole class of tool defects (a retest, a new mxcli release, an audit) — for one CE code or symptom use bin/bug-lookup.sh instead; the ledger is too large to read whole mdl,gate 5,6 ondemand diagnose bootstrap-project skills/bootstrap-project.md Generating a new project's CLAUDE.md — baseline routing plus project-specific facts ba P ondemand spine diff --git a/bin/ready.sh b/bin/ready.sh index 8c2b2240..7d6a5ea6 100755 --- a/bin/ready.sh +++ b/bin/ready.sh @@ -29,7 +29,7 @@ # must-have the per-module must-have list, when the project has one: K ticked of N. Absent: # reported "not present", not counted. # review only under `Interview mode: auto` (bin/interview-mode.sh): a verdict file written -# by a DIFFERENT agent (skills/module-review.md "Independent look"), every page of +# by a DIFFERENT agent (skills/dispatching.md "Independent look before READY"), every page of # the scope scored 1-5 after its last build; a page below 4, or unscored, blocks. # # WHAT IT WRITES (all tracked, all small, no images): @@ -37,6 +37,9 @@ # module, unseen pages, the screenshot names relied on # docs/progress/ready.log one line per run: READY|NOT-READY # — the READY line. An agent never writes it by hand. +# docs/progress/review-packets/.html per module: each page's screenshot beside its target with +# the reviewer's score, under the K-of-N lines. Paths only, no +# images: the page a person opens after an unattended run # docs/progress/look-summary.tsv the git-ignored ledger's summary (look-ledger.sh summary) # then `git commit -- ` (explicit paths: other sessions share the index). # @@ -94,8 +97,17 @@ _epoch() { # ISO-8601 UTC -> epoch, GNU then BSD; 0 when unparseable echo "ready.sh — $SCOPE in $ROOT ($ISO)" # --- multi-app: a per-app scope needs the multi-app resolver; never guess it ------------------- -if [ -n "$APP" ] && ls "$ROOT"/apps/*/*.mpr >/dev/null 2>&1; then - row fail scope "NOT MEASURED — $ROOT holds several apps (apps/*/); per-app scoping of the look ledger is not resolved yet, so no check here can be read for app $APP alone" +# The multi-app resolver (project-bin/_common.sh mxtk_apps) when this toolkit has it; apps/*/*.mpr else. +apps="$( ( PROJECT_ROOT="$ROOT"; . "$TOOLKIT_ROOT/project-bin/_common.sh" >/dev/null 2>&1 + type mxtk_apps >/dev/null 2>&1 && mxtk_apps "$ROOT" 2>/dev/null | cut -f1 ) )" +[ -n "$apps" ] || apps="$(ls -d "$ROOT"/apps/*/ 2>/dev/null | while IFS= read -r d; do ls "$d"*.mpr >/dev/null 2>&1 && basename "$d"; done)" +if [ "$(printf '%s\n' "$apps" | sed '/^$/d' | wc -l | tr -d ' ')" -ge 2 ]; then + # Never "nothing checked": every app is named, and none is READY until the ledger is per app. + for a in $apps; do + [ -z "$APP" ] || [ "$APP" = "$a" ] || continue + row fail "app:$a" "NOT MEASURED — the look ledger is not kept per app yet, so no check here can be read for app $a alone" + done + [ -z "$APP" ] || printf '%s\n' $apps | grep -qx "$APP" || row fail scope "no app $APP here; apps: $(printf '%s ' $apps)" fi # --- gate 5 --------------------------------------------------------------------------------- @@ -153,47 +165,66 @@ if [ -n "$MOD" ]; then for f in st shots; do awk -F'\t' -v m="$MOD" 'index($2, m ".") == 1' "$TMP/$f" > "$TMP/$f.x"; mv "$TMP/$f.x" "$TMP/$f"; done fi -# --- must-have list (when the project has one) ----------------------------------------------- -MH="" -for f in ${MXTK_MUST_HAVES:-} ${MOD:+"$ROOT/architecture/modules/$MOD/must-haves.md"} "$ROOT/docs/must-haves.md"; do - [ -f "$f" ] && { MH="$f"; break; } +# --- must-have mark (skills/must-haves.md § The mark) ------------------------------------------ +# One file per module that holds a page: docs/progress/must-haves/.md (TRACKED, so CI and a +# reviewer see it). Line 1's first "N of M" gives the denominator M; K = the rows "<#> TICKED — ...". +# .claude/loop/must-haves/.md is read as a fallback, flagged: git cannot see it there. +# Owed only once the toolkit ships skills/must-haves.md; before that a missing mark is "not present". +mh_owed=0; [ -f "$TOOLKIT_ROOT/skills/must-haves.md" ] && mh_owed=1 +if [ -n "$MOD" ]; then mh_mods="$MOD" +else mh_mods="$(awk -F'\t' '{ m=$2; sub(/\..*/, "", m); print m }' "$TMP/st" | sort -u)"; fi +mh_k=0; mh_n=0; mh_miss=""; mh_files=""; mh_bad=""; mh_hidden="" +for m in $mh_mods; do + f="$ROOT/docs/progress/must-haves/$m.md" + if [ ! -f "$f" ] && [ -f "$ROOT/.claude/loop/must-haves/$m.md" ]; then f="$ROOT/.claude/loop/must-haves/$m.md"; mh_hidden="$mh_hidden $m"; fi + if [ ! -f "$f" ]; then mh_miss="$mh_miss $m"; continue; fi + mh_files="$mh_files ${f#"$ROOT"/}" + tot="$(head -1 "$f" | tr -d '\r' | grep -oE '[0-9]+ of [0-9]+' | head -1 | sed 's/.* of //')" + [ -n "$tot" ] || { mh_bad="$mh_bad $m"; continue; } + tk="$(tr -d '\r' < "$f" | sed 1d | grep -cE '^[[:space:]]*[0-9]+[.)]?[[:space:]]+TICKED([[:space:]]|$)')" + mh_k=$((mh_k + tk)); mh_n=$((mh_n + tot)) done -if [ -n "$MH" ]; then - # a module's own file counts whole; the shared file counts the lines naming the module (or all) - mh="$(awk -v m="$MOD" -v own="$( [ -n "$MOD" ] && [ "$MH" = "$ROOT/architecture/modules/$MOD/must-haves.md" ] && echo 1 || echo 0)" ' - /^[[:space:]]*[-*][[:space:]]+\[[ xX]\]/ { - if (m != "" && own != 1 && index($0, m) == 0) next - n++; if ($0 ~ /\[[xX]\]/) k++ - } END { print k+0 " " n+0 }' "$MH")" - k="${mh% *}"; n="${mh#* }" - if [ "$n" -eq 0 ]; then row fail must-have "0 of 0 — ${MH#"$ROOT"/} lists no checkbox item${MOD:+ for $MOD}: an empty list proves nothing" - elif [ "$k" -lt "$n" ]; then row fail must-have "$k of $n ticked in ${MH#"$ROOT"/}" - else row ok must-have "$n of $n ticked in ${MH#"$ROOT"/}"; fi +mh_where="docs/progress/must-haves/${MOD:-}.md" +if [ -z "$mh_mods" ]; then + row info must-have "not measured — no built page names a module that owes a mark ($mh_where)" +elif [ -n "$mh_bad" ]; then + row fail must-have "FAULT — line 1 has no \"N of M\" in the mark of:$mh_bad" +elif [ -n "$mh_miss" ] && [ "$mh_owed" = 1 ]; then + row fail must-have "$mh_k of $mh_n ticked; no mark for:$mh_miss (write $mh_where, skills/must-haves.md)" +elif [ -z "$mh_files" ]; then + row info must-have "not present — no mark at $mh_where" +elif [ "$mh_k" -lt "$mh_n" ]; then + row fail must-have "$mh_k of $mh_n rows TICKED in$mh_files${mh_miss:+ (no mark for:$mh_miss)}" else - row info must-have "not present — no must-have list at ${MOD:+architecture/modules/$MOD/must-haves.md or }docs/must-haves.md" + row ok must-have "$mh_n of $mh_n rows TICKED in$mh_files${mh_miss:+ (no mark for:$mh_miss)}" fi +[ -n "$mh_hidden" ] && echo " note must-have mark read from git-ignored .claude/loop/ for:$mh_hidden — move it to docs/progress/must-haves/ so CI and reviewers see it" >&2 # --- independent review, unattended mode only ------------------------------------------------- MODE="$(bash "$TOOLKIT_ROOT/bin/interview-mode.sh" "$ROOT" 2>/dev/null)" RV_DIR="$ROOT/docs/progress/look-reviews" : > "$TMP/scores" +# Review files are read in every mode (the packet shows them); only auto mode requires them. +for f in "$RV_DIR"/*.md; do + [ -f "$f" ] || continue + rvr="$(sed -n 's/^Reviewer:[[:space:]]*//p' "$f" | head -1 | tr -d '\r')" + bld="$(sed -n 's/^Builder:[[:space:]]*//p' "$f" | head -1 | tr -d '\r')" + at="$(sed -n 's/^Reviewed:[[:space:]]*//p' "$f" | head -1 | tr -d '\r' | awk '{print $1}')" + # a self-review is no review: both names present and different, or the file is not read + if [ -z "$rvr" ] || [ -z "$bld" ] || [ "$rvr" = "$bld" ]; then + echo " note review file ${f#"$ROOT"/} ignored: needs distinct Reviewer: and Builder: lines" >&2; continue + fi + ep="$(_epoch "$at")" + # | Page | Screenshot | Target | Score | Note | — columns found by header, page is the first + tr -d '\r' < "$f" | awk -F'|' -v ep="$ep" -v fn="${f#"$ROOT"/}" ' + function cell(i, v) { if (!i) return ""; v=$i; gsub(/^[[:space:]`]+|[[:space:]`]+$/, "", v); gsub(/\t/, " ", v); return v } + sc == 0 && /^\|/ { for (i=2; i 0 && /^\|/ { p=$2; gsub(/[[:space:]`]/, "", p); s=$sc; gsub(/[^0-9]/, "", s) + if (p ~ /^[A-Za-z0-9_]+\.[A-Za-z0-9_]+$/ && s != "") + print p "\t" s "\t" ep "\t" fn "\t" cell(ss) "\t" cell(tg) "\t" cell(nt) }' >> "$TMP/scores" +done if [ "$MODE" = auto ]; then - for f in "$RV_DIR"/*.md; do - [ -f "$f" ] || continue - rvr="$(sed -n 's/^Reviewer:[[:space:]]*//p' "$f" | head -1 | tr -d '\r')" - bld="$(sed -n 's/^Builder:[[:space:]]*//p' "$f" | head -1 | tr -d '\r')" - at="$(sed -n 's/^Reviewed:[[:space:]]*//p' "$f" | head -1 | tr -d '\r' | awk '{print $1}')" - # a self-review is no review: both names present and different, or the file is not read - if [ -z "$rvr" ] || [ -z "$bld" ] || [ "$rvr" = "$bld" ]; then - echo " note review file ${f#"$ROOT"/} ignored: needs distinct Reviewer: and Builder: lines" >&2; continue - fi - ep="$(_epoch "$at")" - # table rows: | Module.Page | screenshot | target | score | note | — Score column found by header - tr -d '\r' < "$f" | awk -F'|' -v ep="$ep" -v fn="${f#"$ROOT"/}" ' - sc == 0 && /^\|/ { for (i=2; i 0 && /^\|/ { p=$2; gsub(/[[:space:]`]/, "", p); s=$sc; gsub(/[^0-9]/, "", s) - if (p ~ /^[A-Za-z0-9_]+\.[A-Za-z0-9_]+$/ && s != "") print p "\t" s "\t" ep "\t" fn }' >> "$TMP/scores" - done if [ ! -s "$TMP/st" ]; then row fail review "NOT MEASURED — no owed page to review${MOD:+ in $MOD}; the reviewer's denominator is the look ledger's pages" else @@ -210,7 +241,7 @@ if [ "$MODE" = auto ]; then END { printf "%d|%d|%d|%d|%s|%s\n", k, n, miss, low, ml, ll }' "$TMP/st" > "$TMP/rv" IFS='|' read -r k n miss low ml ll < "$TMP/rv" # not a tab: IFS whitespace collapses empty fields if [ "$k" = "$n" ]; then row ok review "$n of $n page(s) scored 4+ by an independent reviewer (docs/progress/look-reviews/)" - else row fail review "$k of $n page(s) scored 4+ by an independent reviewer${ml:+ — unscored since last build: $ml}${ll:+ — below 4: $ll}. Dispatch a fresh reviewer (skills/module-review.md \"Independent look\")"; fi + else row fail review "$k of $n page(s) scored 4+ by an independent reviewer${ml:+ — unscored since last build: $ml}${ll:+ — below 4: $ll}. Dispatch a fresh reviewer (skills/dispatching.md \"Independent look before READY\")"; fi fi else row info review "not owed — Interview mode is ${MODE:-steering}, not auto (a human sees the screens)" @@ -250,18 +281,68 @@ mkdir -p "$ROOT/docs/progress" echo "No look ledger in this checkout (.claude/loop/look/ is git-ignored); see docs/progress/look-summary.tsv if committed." echo fi - if [ -n "$MH" ]; then echo "Must-have list: ${MH#"$ROOT"/}"; echo; fi + if [ -n "$mh_files" ]; then echo "Must-have marks:$mh_files"; echo; fi if [ -s "$TMP/scores" ]; then echo "Review files read: $(cut -f4 "$TMP/scores" | sort -u | tr '\n' ' ')" echo fi } > "$ROOT/$EVID_REL" +# --- the review packet: what a person opens after an unattended run --------------------------- +# Per module, each page's screenshot beside its target with the reviewer's score, under the K-of-N +# lines. An index only: it names image paths and commits none, so it renders where the screenshots +# live (the build's checkout) and still reads as a table anywhere else. +# Under review-packets/, not docs/progress/*.html: report-disposition-check.sh reads that glob as test +# reports owing improvement-register rows, and its low scores already block READY here. +PACKET_REL="docs/progress/review-packets/$SLUG.html" +mkdir -p "$ROOT/docs/progress/review-packets" +awk -F'\t' -v root="$ROOT" -v scope="$SCOPE" -v iso="$ISO" -v verdict="$VERDICT" -v rowsf="$TMP/rows.md" \ + -v shotf="$TMP/shots" -v scf="$TMP/scores" ' + function esc(x) { gsub(/&/, "\\&", x); gsub(//, "\\>", x); gsub(/"/, "\\"", x); return x } + function rel(x) { if (index(x, root "/") == 1) return substr(x, length(root) + 2); return x } + function media(x, r) { if (x == "") return "none"; r = "../../../" esc(rel(x)) + if (tolower(x) ~ /\.(png|jpe?g|webp|gif)$/) return "\""
" esc(rel(x)) "
" + return "" esc(rel(x)) "" } + BEGIN { + while ((getline l < shotf) > 0) { split(l, a, "\t"); shot[a[2]] = a[3] } + while ((getline l < scf) > 0) { split(l, a, "\t"); if (!(a[1] in rat) || a[3]+0 > rat[a[1]]) { rat[a[1]]=a[3]+0; rsc[a[1]]=a[2]; rss[a[1]]=a[5]; rtg[a[1]]=a[6]; rnt[a[1]]=a[7] } } + print "" + print "Review packet" + print "

Review packet: " esc(scope) "

" esc(verdict) " · " esc(iso) " · written by bin/ready.sh. Images are not committed: open this file in the build checkout to see them.

" + print "" + while ((getline l < rowsf) > 0) { n = split(l, c, " \\| "); sub(/^\| /, "", c[1]); sub(/ \|$/, "", c[n]) + print "" } + print "
CheckVerdictDetail
" esc(c[1]) "" esc(c[2]) "" esc(c[3]) "
" + } + { m = $2; sub(/\..*/, "", m); if (!(m in seenm)) { ord[++k] = m; seenm[m] = 1 }; pages[m] = pages[m] SUBSEP $2; st[$2] = $1; built[$2] = $3+0 } + END { + if (k == 0) print "

No page recorded as built in this checkout: nothing to show.

" + for (i = 1; i <= k; i++) { + m = ord[i]; np = split(substr(pages[m], 2), pl, SUBSEP); ns = 0; nr = 0 + for (j = 1; j <= np; j++) { p = pl[j]; if (st[p] == "SEEN") ns++; if ((p in rat) && rat[p] >= built[p] && rsc[p]+0 >= 4) nr++ } + print "

" esc(m) "

" ns " of " np " page(s) seen since their last build · " nr " of " np " scored 4+ by an independent reviewer

" + print "" + for (j = 1; j <= np; j++) { p = pl[j] + sh = (p in rss && rss[p] != "") ? root "/" rss[p] : shot[p] + stale = ((p in rat) && rat[p] < built[p]) ? " (before last build)" : "" + sc = (p in rsc) ? rsc[p] stale : "unscored" + cls = (!(p in rsc) || stale != "" || rsc[p]+0 < 4) ? "low" : "" + print "" } + print "
PageScreenshotTargetScoreNote
" esc(p) "
" (st[p] == "SEEN" ? "seen" : "not seen") "
" media(sh) "" media((p in rtg && rtg[p] != "") ? root "/" rtg[p] : "") "" esc(sc) "" esc(rnt[p]) "
" + } + print "" + }' "$TMP/st" > "$ROOT/$PACKET_REL" printf '%s\t%s\t%s\t%s\t%s of %s\n' "$ISO" "$VERDICT" "$SCOPE" "$EVID_REL" "$N_OK" "$N_ROWS" >> "$ROOT/$LOG_REL" -echo "evidence: $EVID_REL · log: $LOG_REL" +echo "evidence: $EVID_REL · packet: $PACKET_REL · log: $LOG_REL" # --- commit the evidence (explicit paths only) -------------------------------------------------- if [ "$COMMIT" = 1 ] && git -C "$ROOT" rev-parse --git-dir >/dev/null 2>&1; then - paths="$EVID_REL $LOG_REL" + paths="$EVID_REL $LOG_REL $PACKET_REL" [ -f "$ROOT/docs/progress/look-summary.tsv" ] && paths="$paths docs/progress/look-summary.tsv" # shellcheck disable=SC2086 if ( cd "$ROOT" && git add -- $paths && git commit -q -m "ready: $SCOPE $VERDICT ($N_OK of $N_ROWS checks)" -- $paths ) >/dev/null 2>&1; then diff --git a/bin/sync-project.sh b/bin/sync-project.sh index 8e74371e..9ed42553 100755 --- a/bin/sync-project.sh +++ b/bin/sync-project.sh @@ -1340,6 +1340,14 @@ if [ "$WIRED" -eq 1 ] && [ ! -f "$PROJECT_DIR/docs/progress/RESUME.md" ]; then CHANGES=$((CHANGES + 1)) fi +# --- 4a1. CI gate: .github/workflows/toolkit-gates.yml, create-if-absent ------------------------ +# The gate an agent cannot skip (bin/ci-gates.sh, on every push and PR). Never overwritten: a copy +# that differs from templates/ci/toolkit-gates.yml is reported, not replaced (bin/lib/ci-template.sh). +if [ "$WIRED" -eq 1 ]; then + . "$TOOLKIT_ROOT/bin/lib/ci-template.sh" + mxtk_ci_install "$TOOLKIT_ROOT" "$PROJECT_DIR" "$DRY_RUN" && CHANGES=$((CHANGES + 1)) +fi + # --- 4b. Verification engine: tests/e2e/ (same contract as the crash net) ---------------- # project-bin/verify-module.sh has been installed by section 4 for weeks and it DRIVES # project-tests/e2e/journey-runner.js — which reached exactly one project, by hand, and that diff --git a/templates/ci/toolkit-gates.yml b/templates/ci/toolkit-gates.yml new file mode 100644 index 00000000..dea2a679 --- /dev/null +++ b/templates/ci/toolkit-gates.yml @@ -0,0 +1,45 @@ +# toolkit-gates.yml — installed by mxcli-project-toolkit (bin/init-project.sh, bin/sync-project.sh). +# Never overwritten once present: edit it freely; sync only reports when it differs from the template. +# +# The gate an agent cannot skip. Everything inside an agent's session can be skipped by the agent +# (not running a gate, `git commit --no-verify`, writing READY by hand). This runs on GitHub, on what +# was pushed, and runs the toolkit's bin/ci-gates.sh at the toolkit commit PROJECT.md pins: +# (a) gate-check for the stage the register claims fails on FAIL +# (b) a READY line that bin/ready.sh did not write fails +# (c) built pages never looked at fail once anything claims done (READY, or Stage 6+) +# (d) model work committed before Stage 4 passed fails (greenfield and --adopt points excepted) +# +# WHAT IT CANNOT SEE — anything not committed: the git-ignored .claude/loop/ ledgers (only the summary +# docs/progress/look-summary.tsv that bin/ready.sh and close-task.sh commit), screenshots, chat, and a +# model changed without exec.sh and without a docs/BUILD-LOG.md row. No Mendix runtime, no mxcli: it +# reads files only, in seconds. +# +# The toolkit repo defaults to the public one; set the repository variable MXTK_TOOLKIT_REPO to use a +# fork (a private fork also needs a read token in the checkout step). +name: toolkit-gates +on: + push: + pull_request: +permissions: + contents: read +jobs: + gates: + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - uses: actions/checkout@v4 + - name: Toolkit at the commit PROJECT.md pins + env: + TOOLKIT_REPO: ${{ vars.MXTK_TOOLKIT_REPO || 'https://github.com/MendixMau/mxcli-project-toolkit.git' }} + run: | + sha="$(sed -n 's/^Toolkit commit:[[:space:]`]*\([0-9a-f]\{7,40\}\).*/\1/p' PROJECT.md | head -1)" + git clone -q --filter=blob:none "$TOOLKIT_REPO" "$RUNNER_TEMP/toolkit" + if [ -n "$sha" ] && git -C "$RUNNER_TEMP/toolkit" checkout -q "$sha" 2>/dev/null \ + && [ -f "$RUNNER_TEMP/toolkit/bin/ci-gates.sh" ]; then + echo "toolkit pinned at $sha" + else + git -C "$RUNNER_TEMP/toolkit" checkout -q master + echo "::warning title=toolkit gates::PROJECT.md pins no usable toolkit commit (or one older than bin/ci-gates.sh); ran master instead" + fi + - name: Toolkit gates + run: bash "$RUNNER_TEMP/toolkit/bin/ci-gates.sh" . diff --git a/tests/wave2/test-ci-gates.sh b/tests/wave2/test-ci-gates.sh new file mode 100755 index 00000000..fd017ba6 --- /dev/null +++ b/tests/wave2/test-ci-gates.sh @@ -0,0 +1,128 @@ +#!/usr/bin/env bash +# test-ci-gates.sh — bin/ci-gates.sh, the CI gate an agent cannot skip, run the way CI runs it: on a +# fresh clone of the committed tree, without the git-ignored .claude/loop/ ledgers. One project tree per +# failure case, plus a clean one and the installer's never-overwrite contract. +# +# Golden input is produced by the real producers, never typed: page names and screenshots from the +# captured lab app (fixtures/look-shots, mxcli v0.28.0), the owed/seen ledgers by look-ledger.sh owe and +# the seen hook, ready.log / evidence / look-summary.tsv by bin/ready.sh, the "## Current stage" line in +# the exact form gate-check.sh derives, and the BUILD-LOG header and row in the shape exec.sh log_build() +# writes (bin/lib/build-log-read.sh quotes a real row: `| 2026-10-21T05:41:52+0000 | `x.mdl` | … |`). +# +# Usage: bash tests/wave2/test-ci-gates.sh [path-to-bin/ci-gates.sh] + +SUBJECT="${1:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/bin/ci-gates.sh}" +case "$SUBJECT" in /*) ;; *) SUBJECT="$PWD/$SUBJECT" ;; esac +TK="$(cd "$(dirname "$SUBJECT")/.." && pwd)" +FXS="$TK/tests/wave2/fixtures/look-shots" +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; } +WORK="$(mktemp -d "${TMPDIR:-/tmp}/citest.XXXXXX")" || exit 2 +trap 'rm -rf "$WORK"' EXIT +echo "== subject: $SUBJECT" +[ -f "$SUBJECT" ] && [ -d "$FXS" ] || { echo "FAIL — subject or fixture missing"; echo "PASS=0 FAIL=1"; exit 1; } +export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@example.invalid GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@example.invalid +export MXTK_NO_GUIDE=1 +G() { git -C "$1" "${@:2}"; } +hook() { printf '{"hook_event_name":"PostToolUse","tool_name":"Read","tool_input":{"file_path":"%s"},"tool_response":{}}' "$1"; } + +# mkproject — wired, three captured pages built through the real owe path +mkproject() { + mkdir -p "$1/bin" "$1/mdlsource" "$1/shots" "$1/docs/progress"; : > "$1/X.mpr" + cp "$TK/project-bin/look-ledger.sh" "$TK/project-bin/_common.sh" "$1/bin/" + printf '# PROJECT.md\n\n## Current stage\n\n**Stage P — Kickoff**, in progress.\n\nEntry mode: %s\n' "$2" > "$1/PROJECT.md" + printf '.claude/loop/\n' > "$1/.gitignore" + mkdir -p "$1/docs/progress/must-haves" # T1's mark, all rows TICKED: READY must not hinge on whether that skill has landed + printf 'Approval · 2 of 2 rows ticked or NOT DONE with a reason · pages 3 of 3\n1 TICKED — inbox lists open requests\n2 TICKED — decide records the decision\n' > "$1/docs/progress/must-haves/Approval.md" + printf 'CREATE PAGE "Approval"."Dashboard"\n(\n Title: %s\n) { }\ncreate or replace page Approval.Approval_Inbox (Title: %s) { }\nCREATE OR MODIFY PAGE Approval.Approval_Decide {\n}\n' "'Dashboard'" "'Approval inbox'" > "$1/mdlsource/20-approval-pages.mdl" + (cd "$1" && bash bin/look-ledger.sh owe "$1/mdlsource/20-approval-pages.mdl" >/dev/null) + G "$1" init -q; G "$1" add -A; G "$1" commit -qm scaffold +} +see_all() { + local s + for s in dashboard-1280:J-DASH-01-step1.png approval_inbox-1280:J-APP-01-step1.png approval_decide-1280:J-APP-01-step2.png; do + cp "$FXS/tests/e2e/artifacts/${s#*:}" "$1/shots/${s%%:*}.png"; hook "$1/shots/${s%%:*}.png" | (cd "$1" && bash bin/look-ledger.sh seen) + done +} +commit_all() { G "$1" add -A; G "$1" commit -qm "$2"; } +ci() { rm -rf "$WORK/ci"; git clone -q "$1" "$WORK/ci"; out="$(bash "$SUBJECT" "$WORK/ci" 2>&1)"; rc=$?; } + +# ── C0: clean — every page seen, READY by ready.sh, greenfield ──────────────────────────── +P="$WORK/c0"; mkproject "$P" greenfield; see_all "$P" +bash "$TK/bin/ready.sh" "$P" >/dev/null 2>&1 || bad "C0: ready.sh did not pass on a clean project" +ci "$P" +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q '^PASS ready' && printf '%s\n' "$out" | grep -q '^PASS look' \ + && ok "C0: clean tree, READY written by ready.sh → CI passes" || bad "C0: rc=$rc $out" +[ ! -e "$WORK/ci/.claude/loop" ] && ok "C0: CI saw no ledger, only the committed summary" || bad "C0: the clone carries the ignored ledger" + +# ── C1 (b): READY written by hand ────────────────────────────────────────────────────────── +printf '# Progress\n\n| Module | State |\n|---|---|\n| Approval | READY |\n' > "$P/docs/progress/STATUS.md"; commit_all "$P" "hand status" +ci "$P" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL ready docs/progress/STATUS.md:5:' \ + && ok "C1: a hand-written READY table cell fails CI, file:line named" || bad "C1: rc=$rc $out" +G "$P" rm -q docs/progress/STATUS.md; G "$P" commit -qm "drop hand status" +# a hand-appended ready.log row with no evidence behind it +printf '2026-11-17T09:00:00Z\tREADY\tmodule Billing\tdocs/progress/ready-2026-11-17-Billing.md\t3 of 3\n' >> "$P/docs/progress/ready.log"; commit_all "$P" "forged log" +ci "$P" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'FAIL ready docs/progress/ready.log: READY module Billing' \ + && ok "C1: a READY row appended to ready.log by hand fails CI" || bad "C1b: rc=$rc $out" +G "$P" checkout -q HEAD~1 -- docs/progress/ready.log; G "$P" commit -qm "undo forged log" + +# ── C2 (c): READY, then a page built and never looked at ─────────────────────────────────── +sleep 1; printf 'CREATE OR MODIFY PAGE Approval.Approval_Archive {\n}\n' > "$P/mdlsource/21-archive.mdl" +(cd "$P" && bash bin/look-ledger.sh owe "$P/mdlsource/21-archive.mdl" >/dev/null && bash bin/look-ledger.sh summary --write >/dev/null) +commit_all "$P" "archive page" +ci "$P" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL look READY (app) while the committed look summary shows 1 built page(s) never looked at: Approval.Approval_Archive' \ + && ok "C2: READY standing over a committed unseen page fails CI" || bad "C2: rc=$rc $out" +# the same tree without a done-claim is a warning mid-build, a FAIL when strict +P2="$WORK/c2b"; mkproject "$P2" greenfield; (cd "$P2" && bash bin/look-ledger.sh summary --write >/dev/null); commit_all "$P2" summary +ci "$P2" +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q '^WARN look 3 built page(s) not looked at yet' && ok "C2: no claim → WARN, exit 0 (mid-build)" || bad "C2b: rc=$rc $out" +rm -rf "$WORK/ci"; git clone -q "$P2" "$WORK/ci"; out="$(MXTK_CI_STRICT_LOOK=1 bash "$SUBJECT" "$WORK/ci" 2>&1)"; rc=$? +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL look 3 built page(s) never looked at (strict)' && ok "C2: MXTK_CI_STRICT_LOOK=1 → FAIL" || bad "C2c: rc=$rc $out" + +# ── C3 (a): the register claims Stage 5 and Stage 5 FAILs ────────────────────────────────── +sed -i.bak 's/^\*\*Stage P — Kickoff\*\*, in progress\./**Stage 5 — Build**, in progress (manual gate: its verdict is a Decisions row, not a file) — gates passed: P, 0, 1, 2, 3, 4 (derived by gate-check on 2026-11-17; re-run it rather than editing this line)./' "$P2/PROJECT.md"; rm -f "$P2/PROJECT.md.bak" +commit_all "$P2" "claims stage 5" +ci "$P2" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL stage the register claims Stage 5 and its gate FAILs: .*3 of 3 built page(s) never looked at' \ + && ok "C3: a claimed stage whose gate FAILs fails CI (from the committed summary)" || bad "C3: rc=$rc $out" + +# ── C4 (d): model work before Stage 4, requirements-driven ───────────────────────────────── +P3="$WORK/c4"; mkproject "$P3" requirements-driven +cat > "$P3/docs/BUILD-LOG.md" <<'LOG' +# Build log — auto-appended by bin/exec.sh + +One line per exec against the model. Written by the script, not by hand, so it is +true even when someone forgets. Read this before assuming the model builds. + +| when (ISO-8601 local) | script | gate | result | detail | +|---|---|---|---|---| +| 2026-10-21T05:41:52+0000 | `20-approval-pages.mdl` | pass | ok | 3 pages | +LOG +commit_all "$P3" "built early" +ci "$P3" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL order model work (1 exec row(s) in docs/BUILD-LOG.md, 1 build script(s) in mdlsource/) while Stage 4 is' \ + && ok "C4: requirements-driven model work before Stage 4 fails CI" || bad "C4: rc=$rc $out" + +bash "$TK/bin/gate-check.sh" --no-html --adopt 5 --reason "joined at build; stages before it were done elsewhere" "$P3" >/dev/null 2>&1 +commit_all "$P3" adopt +ci "$P3" +printf '%s\n' "$out" | grep -q '^PASS order' && ok "C4: an --adopt 5 point clears it" || bad "C4b: $out" + +# ── I: installer — create if absent, never overwrite ─────────────────────────────────────── +. "$TK/bin/lib/ci-template.sh" +P4="$WORK/inst"; mkdir -p "$P4/.github/workflows"; printf 'name: build\n' > "$P4/.github/workflows/build.yml" +m="$(mxtk_ci_install "$TK" "$P4" 0)" +cmp -s "$TK/templates/ci/toolkit-gates.yml" "$P4/.github/workflows/toolkit-gates.yml" && printf '%s' "$m" | grep -q 'existing workflows left untouched: build.yml' \ + && [ "$(cat "$P4/.github/workflows/build.yml")" = "name: build" ] && ok "I1: installed beside the repo's own CI, which is untouched" || bad "I1: $m" +printf '# adapted\n' >> "$P4/.github/workflows/toolkit-gates.yml" +m="$(mxtk_ci_install "$TK" "$P4" 0)"; tail -1 "$P4/.github/workflows/toolkit-gates.yml" | grep -q '^# adapted' && printf '%s' "$m" | grep -q '^Kept:' \ + && ok "I2: an adapted copy is kept and reported, never overwritten" || bad "I2: $m" +P5="$WORK/inst2"; mkdir -p "$P5"; m="$(mxtk_ci_install "$TK" "$P5" 1)" +[ ! -e "$P5/.github" ] && printf '%s' "$m" | grep -q '^Would create' && ok "I3: dry run writes nothing" || bad "I3: $m" + +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] diff --git a/tests/wave2/test-ready.sh b/tests/wave2/test-ready.sh index 4b241d78..b137e1ed 100755 --- a/tests/wave2/test-ready.sh +++ b/tests/wave2/test-ready.sh @@ -35,6 +35,8 @@ mkproject() { cp "$TK/project-bin/look-ledger.sh" "$TK/project-bin/_common.sh" "$1/bin/" printf '# PROJECT.md\n\n## Current stage\n\n**Stage P — Kickoff**, in progress.\n\n%s\n' "${3:+Interview mode: $3}" > "$1/PROJECT.md" printf '.claude/loop/\n' > "$1/.gitignore" + mkdir -p "$1/docs/progress/must-haves" # T1's mark, all rows TICKED: READY must not hinge on whether that skill has landed + printf 'Approval · 2 of 2 rows ticked or NOT DONE with a reason · pages 3 of 3\n1 TICKED — inbox lists open requests\n2 TICKED — decide records the decision\n' > "$1/docs/progress/must-haves/Approval.md" cat > "$1/mdlsource/20-approval-pages.mdl" <<'MDL' CREATE PAGE "Approval"."Dashboard" ( @@ -75,11 +77,11 @@ see "$P" dashboard-1280 J-DASH-01-step1.png run "$P" [ "$rc" = 1 ] && ok "R1: 1 of 3 seen → exit 1" || bad "R1: rc=$rc" printf '%s\n' "$out" | grep -q 'FAIL look 1 of 3 page(s) looked at' && ok "R1: look row says 1 of 3" || bad "R1: $out" -printf '%s\n' "$out" | grep -q '^NOT-READY — 0 of 2 checks pass' && ok "R1: verdict NOT-READY with its denominator" || bad "R1: verdict: $out" +printf '%s\n' "$out" | grep -q '^NOT-READY — 1 of 3 checks pass' && ok "R1: verdict NOT-READY with its denominator" || bad "R1: verdict: $out" ev="$(ls "$P"/docs/progress/ready-*-app.md 2>/dev/null | head -1)" [ -n "$ev" ] && grep -q '^- Approval.Approval_Inbox' "$ev" && grep -q 'Approval.Dashboard: dashboard-1280.png' "$ev" \ && ok "R1: evidence names the unseen pages and the screenshot relied on" || bad "R1: evidence $(cat "$ev" 2>/dev/null)" -G "$P" ls-files --error-unmatch docs/progress/ready.log docs/progress/look-summary.tsv "${ev#"$P"/}" >/dev/null 2>&1 \ +G "$P" ls-files --error-unmatch docs/progress/ready.log docs/progress/look-summary.tsv docs/progress/review-packets/app.html "${ev#"$P"/}" >/dev/null 2>&1 \ && ok "R1: evidence, ready.log and look-summary.tsv are committed" || bad "R1: not committed: $(G "$P" status --short)" grep -q '^as-of.*owed 3 seen 1 unseen 2' "$P/docs/progress/look-summary.tsv" && ok "R1: look-summary.tsv says owed 3 seen 1" || bad "R1: summary $(cat "$P/docs/progress/look-summary.tsv")" @@ -97,7 +99,7 @@ see_all "$P" run "$P" [ "$rc" = 0 ] && ok "R3: all 3 seen → READY exit 0" || bad "R3: rc=$rc $out" tail -1 "$P/docs/progress/ready.log" | grep -q ' READY app ' && ok "R3: ready.log carries the READY line" || bad "R3: log $(tail -1 "$P/docs/progress/ready.log")" -G "$P" log -1 --format=%s | grep -q '^ready: app READY (2 of 2 checks)' && ok "R3: committed as 'ready: app READY'" || bad "R3: $(G "$P" log -1 --format=%s)" +G "$P" log -1 --format=%s | grep -q '^ready: app READY (3 of 3 checks)' && ok "R3: committed as 'ready: app READY'" || bad "R3: $(G "$P" log -1 --format=%s)" grep -q '^as-of.*owed 3 seen 3 unseen 0' "$P/docs/progress/look-summary.tsv" && ok "R3: summary refreshed to 3 of 3" || bad "R3: summary" # ── R4: unattended (auto) needs an independent review ────────────────────────────────────── @@ -129,11 +131,21 @@ run "$P" --no-commit run "$P" --module Nope --no-commit [ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'look NOT MEASURED' && ok "R5: a module with no built page is NOT MEASURED, never green" || bad "R5: $out" -# ── R6: must-have list, when present ─────────────────────────────────────────────────────── -mkdir -p "$P/architecture/modules/Approval" -printf '# Must-haves — Approval\n- [x] inbox lists open requests\n- [ ] decide sends a notification\n' > "$P/architecture/modules/Approval/must-haves.md" +# ── R6: the must-have mark (skills/must-haves.md § The mark), on its tracked path ────────── +mkdir -p "$P/docs/progress/must-haves" +printf 'Approval · 3 of 3 rows ticked or NOT DONE with a reason · pages 3 of 3\n1 TICKED — inbox lists open requests\n2 TICKED — decide records the decision\n3 NOT DONE — notification: no mail server in the lab\n' > "$P/docs/progress/must-haves/Approval.md" run "$P" --module Approval --no-commit -[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'must-have 1 of 2 ticked' && ok "R6: an unticked must-have blocks, 1 of 2" || bad "R6: $out" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'must-have 2 of 3 rows TICKED' && ok "R6: a NOT DONE row blocks READY, 2 of 3 (denominator from line 1)" || bad "R6: $out" +printf 'Approval · rows not counted yet\n1 TICKED — x\n' > "$P/docs/progress/must-haves/Approval.md" +run "$P" --module Approval --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'must-have FAULT — line 1 has no "N of M"' && ok "R6: a mark whose line 1 states no N of M is a FAULT, not a pass" || bad "R6: $out" +rm -f "$P/docs/progress/must-haves/Approval.md" + +# ── R6b: the review packet — paths, never images ─────────────────────────────────────────── +pk="$P/docs/progress/review-packets/Approval.html" +[ -f "$pk" ] && grep -q 'src="../../../shots/dashboard-1280.png"' "$pk" && grep -q '3 of 3 page(s) seen since their last build' "$pk" \ + && ! ls "$P"/docs/progress/review-packets/*.png >/dev/null 2>&1 \ + && ok "R6b: the packet lists each page's screenshot path and K of N, and holds no image" || bad "R6b: packet $(head -c 400 "$pk" 2>/dev/null)" # ── R7: no ledger at all → NOT MEASURED ──────────────────────────────────────────────────── P="$WORK/r7"; mkdir -p "$P"; printf '# PROJECT.md\n' > "$P/PROJECT.md" From e1a7ac61370748dad37154c18c99194f6869ee41 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 03:33:26 +0000 Subject: [PATCH 3/8] fix(ready, ci): READY needs stages P-4 passed; must-have waiver; visible look count ready.sh adds a position row: stages P and 0-4 must each be PASS, WAIVED (an --adopt point) or not owed by the entry mode, from one read-only full gate-check run. A field run wrote READY with Stage P never passed; a look accounted for in Stage 5 is not enough on its own. ready.sh also honours `Waived obligation must-haves[/]` and shows an ADVISORY obligation verdict without blocking. ci-gates.sh prints "look: K of N pages seen" on every look line, as a warning when it does not fail. Fixtures join at Build through gate-check's own --waive P / --adopt 5; new R8: Stage P not passed, every page seen -> NOT READY. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y --- CHANGELOG.md | 2 +- bin/ci-gates.sh | 12 +++++++----- bin/ready.sh | 32 ++++++++++++++++++++++++++++++-- tests/wave2/test-ci-gates.sh | 10 ++++++---- tests/wave2/test-ready.sh | 17 +++++++++++++++-- 5 files changed, 59 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c0e89e35..db3c3f46 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,7 +18,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m `docs/RELEASE-PROCESS.md` §8. ## Unreleased -- new(ci, ready): **the gate an agent cannot skip, and a READY nobody can hand-write.** `bin/ci-gates.sh` runs in the project's own CI (`.github/workflows/toolkit-gates.yml`, from `templates/ci/`, installed by `init-project.sh` and `sync-project.sh` create-if-absent, never overwritten, beside any existing CI) on every push and PR, over the committed tree at the pinned toolkit commit: the claimed stage's gate FAILs, a READY line `bin/ready.sh` did not write (`bin/lib/ready-claims.sh`, also a Stage 6 FAIL), committed unseen pages under a done-claim, and model work before Stage 4 all fail it, in seconds, with no Mendix runtime. `ready.sh` now writes a review packet (`docs/progress/review-packets/.html`: screenshot beside target with the reviewer's score, paths only), reads the must-have mark on its tracked path `docs/progress/must-haves/.md`, and names every app of a multi-app project instead of checking nothing (field runs, 2026-11: agents skip in-session gates; a hand-written READY graded a 0-of-78-looked build done) — two unattended field runs +- new(ci, ready): **the gate an agent cannot skip, and a READY nobody can hand-write.** `bin/ci-gates.sh` runs in the project's own CI (`.github/workflows/toolkit-gates.yml`, from `templates/ci/`, installed by `init-project.sh` and `sync-project.sh` create-if-absent, never overwritten, beside any existing CI) on every push and PR, over the committed tree at the pinned toolkit commit: the claimed stage's gate FAILs, a READY line `bin/ready.sh` did not write (`bin/lib/ready-claims.sh`, also a Stage 6 FAIL), committed unseen pages under a done-claim, and model work before Stage 4 all fail it (otherwise it prints `look: K of N pages seen` as a visible warning), in seconds, with no Mendix runtime. `ready.sh` now also requires the position (stages P and 0–4 PASS, WAIVED or not owed by the entry mode: a field run wrote READY with Stage P never passed), honours `Waived obligation must-haves`, writes a review packet (`docs/progress/review-packets/.html`: screenshot beside target with the reviewer's score, paths only), reads the must-have mark on its tracked path `docs/progress/must-haves/.md`, and names every app of a multi-app project instead of checking nothing (field runs, 2026-11: agents skip in-session gates; a hand-written READY graded a 0-of-78-looked build done) — two unattended field runs - new(ready, look-ledger, gate-check): **READY is earned through one command, and its evidence is in git.** `bin/ready.sh [--module M | --app A]` runs `gate-check.sh` Stage 5 (Stage 6 once the register is there), the look ledger, the must-have list when present and, under `Interview mode: auto`, an independent reviewer's verdict file (`docs/progress/look-reviews/`, every page scored 1–5 after its last build, below 4 blocks); it prints K of N per check, exits non-zero unless all pass, writes the READY line to `docs/progress/ready.log` itself and commits `docs/progress/ready--.md` with explicit paths. `look-ledger.sh summary --write` puts the git-ignored ledger into a tracked `docs/progress/look-summary.tsv` (written by `ready.sh` and every `close-task.sh`), and `gate-check.sh` Stage 5 reads it when the ledger is absent, so a copy of the repo reports FAIL instead of MANUAL. `dispatching.md` gains the reviewer dispatch rule; the `ready` routing row reaches `CLAUDE.local.md` and every agent stub (field runs, 2026-11: a build wrote READY by hand with 0 of ~78 pages looked at; another marked itself ready 3 hours early with no review pass) — two unattended field runs - fix(status, agents): **`bin/status.sh` WATCHes an Agent Editor project whose extension module does not travel with the model (#778)**: Studio Pro reads `agenteditor.agent` from `modules/AgentEditor.mxmodule` beside the `.mpr`, and a sync of only `.mpr` + `mprcontents/` left it out of git, so the builder stayed green while every colleague's clone showed `unknown type 'agenteditor.agent'`. Two WATCH items, warn-only (exit code and every verdict unchanged), local files and `git ls-files` only, root and `app/` layouts: a `modules/*.mxmodule` git does not track, and a project using Agent Editor (`AgentEditorCommons` in `analysis/app-facts/inventory.json`, or a `create agent` statement in `mdlsource/`) with no `modules/AgentEditor*.mxmodule`. `skills/mendix-agents.md` (on demand) gains the rule plus the note that the GenAI key, `Encryption.EncryptionKey` and `GenAICommons.StoreTraces` belong in the Default configuration, not a start script. Empty agent references / missing model version (mendixlabs/mxcli#1867) and unset runtime constants are not checked: they need a captured agent document (option 2). Fixture `tests/wave2/test-status-extension-modules.sh` — field report from an Agent Editor project, Nov 2026 - fix(permissions): **exec approval `auto` no longer prompts on every exec's mode check, and `install-claude-hooks.sh` no longer runs unprompted (#782)**: `install-claude-permissions.sh` now allow-lists `exec-approval.sh`, `interview-mode.sh` and `permission-profile.sh`, so their reads run without a prompt, while their only writers, `--set` and `--clear`, prompt through six `ask` rules (`Bash(*exec-approval.sh*--set*)`, …) in the shared settings under every profile (an ask rule beats an allow rule, `full`'s bare `Bash` included). `install-claude-hooks.sh`, which writes `~/.claude/settings.json` for every repo on the machine, gets no allow rule. A re-install adds the ask rules and removes the hooks-installer rule an earlier install added (sidecar); entries someone else wrote are untouched, and `sync-project.sh`'s permission check names both until then. Field run on claude 2.1.325: the read ran, `--set auto` and `--clear` were refused, and with the ask rules removed `--set` ran — from the two independent reviews of #781 diff --git a/bin/ci-gates.sh b/bin/ci-gates.sh index ca42d004..3c9374d9 100755 --- a/bin/ci-gates.sh +++ b/bin/ci-gates.sh @@ -90,8 +90,10 @@ late=0; case "$claim" in 6|7) late=1 ;; esac if [ -s "$SUMM" ]; then unseen="$(awk -F'\t' '$1 == "unseen" { print $2 }' "$SUMM")" asof="$(awk -F'\t' '$1 == "as-of" { print $2 " (" $3 ", " $4 ")"; exit }' "$SUMM")" + # "look: K of N pages seen" — shown on every run that does not fail, so the number is in front of people + kn="$(awk -F'\t' '$1 == "as-of" { o=$3; s=$4; sub(/^owed /, "", o); sub(/^seen /, "", s); print "look: " s " of " o " pages seen"; exit }' "$SUMM")" if [ -z "$unseen" ]; then - say PASS look "committed look summary: no built page unseen, as of $asof" + say PASS look "$kn — committed look summary, as of ${asof%% (*}" else n="$(printf '%s\n' "$unseen" | wc -l | tr -d ' ')" hit="" @@ -105,10 +107,10 @@ if [ -s "$SUMM" ]; then $claims EOF list="$(printf '%s\n' "$unseen" | head -8 | tr '\n' ' ')" - if [ -n "$hit" ]; then say FAIL look "READY ($hit) while the committed look summary shows $n built page(s) never looked at: $list" - elif [ "$late" = 1 ]; then say FAIL look "the register is at Stage $claim while $n built page(s) were never looked at: $list" - elif [ "${MXTK_CI_STRICT_LOOK:-0}" = 1 ]; then say FAIL look "$n built page(s) never looked at (strict): $list" - else say WARN look "$n built page(s) not looked at yet (as of $asof): $list — fine mid-build, a FAIL once READY is claimed"; fi + if [ -n "$hit" ]; then say FAIL look "$kn — READY ($hit) while the committed look summary shows $n built page(s) never looked at: $list" + elif [ "$late" = 1 ]; then say FAIL look "$kn — the register is at Stage $claim while $n built page(s) were never looked at: $list" + elif [ "${MXTK_CI_STRICT_LOOK:-0}" = 1 ]; then say FAIL look "$kn — $n built page(s) never looked at (strict): $list" + else say WARN look "$kn — $n built page(s) not looked at yet (as of ${asof%% (*}): $list — fine mid-build, a FAIL once READY or Stage 6 is claimed"; fi fi elif [ -n "$claims" ]; then say FAIL look "READY claimed ($(printf '%s' "$claims" | tr '\n' ';')) but no docs/progress/look-summary.tsv is committed: the look behind the claim is invisible" diff --git a/bin/ready.sh b/bin/ready.sh index 7d6a5ea6..d301be7d 100755 --- a/bin/ready.sh +++ b/bin/ready.sh @@ -19,6 +19,7 @@ # # THE CHECKS (each row prints K of N; a row nobody measured is NOT MEASURED and fails — never # green-by-absence): +# position stages P and 0-4 each PASS, WAIVED (an --adopt point) or not owed by the entry mode. # gate 5 gate-check.sh 5. PASS/WAIVED, or MANUAL with the look accounted for (Stage 5 # is MANUAL by design once the look is clear — its other half is a Decisions row). # gate 6 only when the register's "## Current stage" line is at Stage 6 or later. @@ -110,8 +111,21 @@ if [ "$(printf '%s\n' "$apps" | sed '/^$/d' | wc -l | tr -d ' ')" -ge 2 ]; then [ -z "$APP" ] || printf '%s\n' $apps | grep -qx "$APP" || row fail scope "no app $APP here; apps: $(printf '%s ' $apps)" fi -# --- gate 5 --------------------------------------------------------------------------------- +# --- position: every stage before Build passed, waived (adopted) or not owed by the entry mode --- +# A field run wrote READY with no stage past P ever passed (pinned gate-check: Stage P FAIL). A look +# accounted for in Stage 5 is not enough if the planning it builds on never happened. One full, +# read-only run (--no-html writes nothing) gives every stage's verdict. GC="$TOOLKIT_ROOT/bin/gate-check.sh" +GFULL="$(bash "$GC" --no-html "$ROOT" 2>&1)" +pk=0; pbad="" +for st in P 0 1 2 3 4; do + v="$(printf '%s\n' "$GFULL" | grep -m1 "^Stage $st (" | sed "s/^Stage $st ([^)]*): //" | awk '{print $1}')" + case "$v" in PASS|WAIVED|N/A) pk=$((pk+1)) ;; *) pbad="$pbad${pbad:+, }$st ${v:-unknown}" ;; esac +done +if [ -z "$pbad" ]; then row ok position "6 of 6 stages before Build PASS or WAIVED (P, 0-4)" +else row fail position "$pk of 6 stages before Build PASS or WAIVED — not: $pbad. Pass them, or record where this project joined (gate-check.sh --adopt 5 --reason \"...\")"; fi + +# --- gate 5 --------------------------------------------------------------------------------- g5="$(MXTK_LOOK_MODULE="$MOD" bash "$GC" --no-html "$ROOT" 5 2>&1)"; rc5=$? l5="$(printf '%s\n' "$g5" | grep -m1 '^Stage 5 (' | sed 's/^Stage 5 ([^)]*): //')" v5="$(printf '%s' "$l5" | awk '{print $1}')" @@ -185,18 +199,32 @@ for m in $mh_mods; do mh_k=$((mh_k + tk)); mh_n=$((mh_n + tot)) done mh_where="docs/progress/must-haves/${MOD:-}.md" +# The standard obligation waiver (`Waived obligation must-haves[/]: `) clears a +# missing mark; an ADVISORY verdict from the obligation check (a project past Stage 5 at sync) shows +# without blocking. +mh_waived=""; mh_left="" +for m in $mh_miss; do + if [ -f "$REG" ] && grep -qiE "^[-*[:space:]]*Waived obligation must-haves(/$m)?[[:space:]]*:" "$REG"; then mh_waived="$mh_waived $m" + else mh_left="$mh_left $m"; fi +done +mh_miss="$mh_left" +mh_adv="$(printf '%s\n' "$GFULL" | grep -m1 -E '^Obligation must-haves[[:space:]]+ADVISORY')" if [ -z "$mh_mods" ]; then row info must-have "not measured — no built page names a module that owes a mark ($mh_where)" elif [ -n "$mh_bad" ]; then row fail must-have "FAULT — line 1 has no \"N of M\" in the mark of:$mh_bad" +elif [ -n "$mh_miss" ] && [ "$mh_owed" = 1 ] && [ -n "$mh_adv" ]; then + row info must-have "ADVISORY (not blocking) — no mark for:$mh_miss; $(printf '%s' "$mh_adv" | sed 's/^Obligation must-haves[[:space:]]*//' | cut -c1-120)" elif [ -n "$mh_miss" ] && [ "$mh_owed" = 1 ]; then row fail must-have "$mh_k of $mh_n ticked; no mark for:$mh_miss (write $mh_where, skills/must-haves.md)" +elif [ -z "$mh_files" ] && [ -n "$mh_waived" ]; then + row ok must-have "WAIVED — Waived obligation must-haves in PROJECT.md for:$mh_waived" elif [ -z "$mh_files" ]; then row info must-have "not present — no mark at $mh_where" elif [ "$mh_k" -lt "$mh_n" ]; then row fail must-have "$mh_k of $mh_n rows TICKED in$mh_files${mh_miss:+ (no mark for:$mh_miss)}" else - row ok must-have "$mh_n of $mh_n rows TICKED in$mh_files${mh_miss:+ (no mark for:$mh_miss)}" + row ok must-have "$mh_n of $mh_n rows TICKED in$mh_files${mh_miss:+ (no mark for:$mh_miss)}${mh_waived:+ (waived:$mh_waived)}" fi [ -n "$mh_hidden" ] && echo " note must-have mark read from git-ignored .claude/loop/ for:$mh_hidden — move it to docs/progress/must-haves/ so CI and reviewers see it" >&2 diff --git a/tests/wave2/test-ci-gates.sh b/tests/wave2/test-ci-gates.sh index fd017ba6..504dfe7e 100755 --- a/tests/wave2/test-ci-gates.sh +++ b/tests/wave2/test-ci-gates.sh @@ -50,9 +50,11 @@ ci() { rm -rf "$WORK/ci"; git clone -q "$1" "$WORK/ci"; out="$(bash "$SUBJECT" " # ── C0: clean — every page seen, READY by ready.sh, greenfield ──────────────────────────── P="$WORK/c0"; mkproject "$P" greenfield; see_all "$P" +bash "$TK/bin/gate-check.sh" --no-html --waive P --reason "fixture: kickoff held elsewhere" "$P" >/dev/null 2>&1 # READY needs P-4 passed or waived +commit_all "$P" "kickoff waived" bash "$TK/bin/ready.sh" "$P" >/dev/null 2>&1 || bad "C0: ready.sh did not pass on a clean project" ci "$P" -[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q '^PASS ready' && printf '%s\n' "$out" | grep -q '^PASS look' \ +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q '^PASS ready' && printf '%s\n' "$out" | grep -q '^PASS look look: 3 of 3 pages seen' \ && ok "C0: clean tree, READY written by ready.sh → CI passes" || bad "C0: rc=$rc $out" [ ! -e "$WORK/ci/.claude/loop" ] && ok "C0: CI saw no ledger, only the committed summary" || bad "C0: the clone carries the ignored ledger" @@ -74,14 +76,14 @@ sleep 1; printf 'CREATE OR MODIFY PAGE Approval.Approval_Archive {\n}\n' > "$P/m (cd "$P" && bash bin/look-ledger.sh owe "$P/mdlsource/21-archive.mdl" >/dev/null && bash bin/look-ledger.sh summary --write >/dev/null) commit_all "$P" "archive page" ci "$P" -[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL look READY (app) while the committed look summary shows 1 built page(s) never looked at: Approval.Approval_Archive' \ +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL look look: 3 of 4 pages seen — READY (app) while the committed look summary shows 1 built page(s) never looked at: Approval.Approval_Archive' \ && ok "C2: READY standing over a committed unseen page fails CI" || bad "C2: rc=$rc $out" # the same tree without a done-claim is a warning mid-build, a FAIL when strict P2="$WORK/c2b"; mkproject "$P2" greenfield; (cd "$P2" && bash bin/look-ledger.sh summary --write >/dev/null); commit_all "$P2" summary ci "$P2" -[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q '^WARN look 3 built page(s) not looked at yet' && ok "C2: no claim → WARN, exit 0 (mid-build)" || bad "C2b: rc=$rc $out" +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q '^WARN look look: 0 of 3 pages seen — 3 built page(s) not looked at yet' && ok "C2: no claim → WARN, exit 0 (mid-build)" || bad "C2b: rc=$rc $out" rm -rf "$WORK/ci"; git clone -q "$P2" "$WORK/ci"; out="$(MXTK_CI_STRICT_LOOK=1 bash "$SUBJECT" "$WORK/ci" 2>&1)"; rc=$? -[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL look 3 built page(s) never looked at (strict)' && ok "C2: MXTK_CI_STRICT_LOOK=1 → FAIL" || bad "C2c: rc=$rc $out" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL look look: 0 of 3 pages seen — 3 built page(s) never looked at (strict)' && ok "C2: MXTK_CI_STRICT_LOOK=1 → FAIL" || bad "C2c: rc=$rc $out" # ── C3 (a): the register claims Stage 5 and Stage 5 FAILs ────────────────────────────────── sed -i.bak 's/^\*\*Stage P — Kickoff\*\*, in progress\./**Stage 5 — Build**, in progress (manual gate: its verdict is a Decisions row, not a file) — gates passed: P, 0, 1, 2, 3, 4 (derived by gate-check on 2026-11-17; re-run it rather than editing this line)./' "$P2/PROJECT.md"; rm -f "$P2/PROJECT.md.bak" diff --git a/tests/wave2/test-ready.sh b/tests/wave2/test-ready.sh index b137e1ed..d8a5890c 100755 --- a/tests/wave2/test-ready.sh +++ b/tests/wave2/test-ready.sh @@ -47,6 +47,12 @@ CREATE OR MODIFY PAGE Approval.Approval_Decide { } MDL (cd "$1" && bash bin/look-ledger.sh owe "$1/mdlsource/20-approval-pages.mdl" >/dev/null) + # The position READY needs: stages P-4 PASS or WAIVED. The fixture joins at Build the way a real + # project records it, through gate-check's own --waive / --adopt (NOADOPT=1 skips this). + if [ -z "${NOADOPT:-}" ]; then + bash "$TK/bin/gate-check.sh" --no-html --waive P --reason "fixture: kickoff held elsewhere" "$1" >/dev/null 2>&1 + bash "$TK/bin/gate-check.sh" --no-html --adopt 5 --reason "fixture: joins at build" "$1" >/dev/null 2>&1 + fi G "$1" init -q; G "$1" add -A; G "$1" commit -qm scaffold } # see — a screenshot named for its page, opened by the agent @@ -77,7 +83,7 @@ see "$P" dashboard-1280 J-DASH-01-step1.png run "$P" [ "$rc" = 1 ] && ok "R1: 1 of 3 seen → exit 1" || bad "R1: rc=$rc" printf '%s\n' "$out" | grep -q 'FAIL look 1 of 3 page(s) looked at' && ok "R1: look row says 1 of 3" || bad "R1: $out" -printf '%s\n' "$out" | grep -q '^NOT-READY — 1 of 3 checks pass' && ok "R1: verdict NOT-READY with its denominator" || bad "R1: verdict: $out" +printf '%s\n' "$out" | grep -q '^NOT-READY — 2 of 4 checks pass' && ok "R1: verdict NOT-READY with its denominator" || bad "R1: verdict: $out" ev="$(ls "$P"/docs/progress/ready-*-app.md 2>/dev/null | head -1)" [ -n "$ev" ] && grep -q '^- Approval.Approval_Inbox' "$ev" && grep -q 'Approval.Dashboard: dashboard-1280.png' "$ev" \ && ok "R1: evidence names the unseen pages and the screenshot relied on" || bad "R1: evidence $(cat "$ev" 2>/dev/null)" @@ -99,7 +105,7 @@ see_all "$P" run "$P" [ "$rc" = 0 ] && ok "R3: all 3 seen → READY exit 0" || bad "R3: rc=$rc $out" tail -1 "$P/docs/progress/ready.log" | grep -q ' READY app ' && ok "R3: ready.log carries the READY line" || bad "R3: log $(tail -1 "$P/docs/progress/ready.log")" -G "$P" log -1 --format=%s | grep -q '^ready: app READY (3 of 3 checks)' && ok "R3: committed as 'ready: app READY'" || bad "R3: $(G "$P" log -1 --format=%s)" +G "$P" log -1 --format=%s | grep -q '^ready: app READY (4 of 4 checks)' && ok "R3: committed as 'ready: app READY'" || bad "R3: $(G "$P" log -1 --format=%s)" grep -q '^as-of.*owed 3 seen 3 unseen 0' "$P/docs/progress/look-summary.tsv" && ok "R3: summary refreshed to 3 of 3" || bad "R3: summary" # ── R4: unattended (auto) needs an independent review ────────────────────────────────────── @@ -152,5 +158,12 @@ P="$WORK/r7"; mkdir -p "$P"; printf '# PROJECT.md\n' > "$P/PROJECT.md" run "$P" --no-commit [ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'look NOT MEASURED' && ok "R7: no ledger → NOT READY, look NOT MEASURED" || bad "R7: rc=$rc $out" +# ── R8: every page seen, but no stage before Build ever passed → NOT READY (field run, 2026-11) ─ +P="$WORK/r8"; NOADOPT=1 mkproject "$P" root; see_all "$P" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'FAIL position .* of 6 stages before Build PASS or WAIVED — not: P ' \ + && printf '%s\n' "$out" | grep -q 'PASS look 3 of 3' \ + && ok "R8: Stage P not passed, all 3 pages seen → NOT READY on position" || bad "R8: rc=$rc $out" + echo "PASS=$PASS FAIL=$FAIL" [ "$FAIL" -eq 0 ] From 15618fd66a104e6a1d6a7771a162f5f61d3e6e2a Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 03:34:00 +0000 Subject: [PATCH 4/8] fix(tests): fixture git identity t@t, as the other fixtures use (leak guard blocks email-shaped strings) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y --- tests/wave2/test-ci-gates.sh | 2 +- tests/wave2/test-ready.sh | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/wave2/test-ci-gates.sh b/tests/wave2/test-ci-gates.sh index 504dfe7e..48136f09 100755 --- a/tests/wave2/test-ci-gates.sh +++ b/tests/wave2/test-ci-gates.sh @@ -22,7 +22,7 @@ WORK="$(mktemp -d "${TMPDIR:-/tmp}/citest.XXXXXX")" || exit 2 trap 'rm -rf "$WORK"' EXIT echo "== subject: $SUBJECT" [ -f "$SUBJECT" ] && [ -d "$FXS" ] || { echo "FAIL — subject or fixture missing"; echo "PASS=0 FAIL=1"; exit 1; } -export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@example.invalid GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@example.invalid +export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@t GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@t export MXTK_NO_GUIDE=1 G() { git -C "$1" "${@:2}"; } hook() { printf '{"hook_event_name":"PostToolUse","tool_name":"Read","tool_input":{"file_path":"%s"},"tool_response":{}}' "$1"; } diff --git a/tests/wave2/test-ready.sh b/tests/wave2/test-ready.sh index d8a5890c..1e51c12e 100755 --- a/tests/wave2/test-ready.sh +++ b/tests/wave2/test-ready.sh @@ -24,9 +24,9 @@ echo "== subject: $SUBJECT" [ -f "$SUBJECT" ] && [ -d "$FXS" ] || { echo "FAIL — subject or fixture missing"; echo "PASS=0 FAIL=1"; exit 1; } # ready.sh commits with the caller's git identity; a CI runner has none -export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@example.invalid GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@example.invalid +export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@t GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@t hook() { printf '{"hook_event_name":"PostToolUse","tool_name":"Read","tool_input":{"file_path":"%s"},"tool_response":{}}' "$1"; } -G() { git -C "$1" -c user.email=t@example.invalid -c user.name=t "${@:2}"; } +G() { git -C "$1" -c user.email=t@t -c user.name=t "${@:2}"; } # mkproject [mode] — a wired project: bin/ installed, three captured pages built mkproject() { From e10cf3127d7a38c3f48383f59ccdf2dd33e5d29d Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 03:40:24 +0000 Subject: [PATCH 5/8] =?UTF-8?q?fix(ready,=20ci):=20field-run=20defects=20?= =?UTF-8?q?=E2=80=94=20heading=20READY=20claims,=20bounded=20steps,=20per-?= =?UTF-8?q?app=20scripts?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit From the parent's field run on a six-app copy: - ready-claims: READY as the first word counts whatever follows it, so "# READY FOR BENCH (fix round 6, ...)" (five times in a field status file) is a claim. Captured shape added to test-ci-gates C1. - ready.sh: every external step names itself on stderr, runs with stdin closed under a watchdog (MXTK_READY_STEP_TIMEOUT, default 120 s) and reports NOT MEASURED when killed; the step subshell clears the EXIT trap so a kill cannot delete $TMP. Not reproduced here (6 apps, 430 BUILD-LOG rows: 1.5 s); the progress lines will name the step on the re-run. R9. - ci-gates order: counts apps//mdlsource/ via mxtk_apps or the glob. C5. Look now WARNs when exec rows exist but no look summary is committed. - gate-check Stage 5: rows="" — unbound under set -u on a project with no ledger and no summary, which emptied the verdict (regression from 3e9302c). Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y --- CHANGELOG.md | 1 + bin/ci-gates.sh | 23 +++++++++++++--- bin/gate-check.sh | 2 +- bin/lib/ready-claims.sh | 9 ++++--- bin/ready.sh | 52 +++++++++++++++++++++++++++++------- tests/wave2/test-ci-gates.sh | 18 +++++++++++++ tests/wave2/test-ready.sh | 8 ++++++ 7 files changed, 96 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index db3c3f46..5facb1ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m `docs/RELEASE-PROCESS.md` §8. ## Unreleased +- fix(ready, ci, gate-check): **field run of the CI gate and `ready.sh` on a six-app copy (#788)**: a hand-written `# READY FOR BENCH (fix round 6, …)` heading passed the `ready` check (READY followed by words now counts; heading markup stripped); `ready.sh` ran over 300 s with no output (every external step now names itself, has stdin closed and a watchdog, `MXTK_READY_STEP_TIMEOUT`, default 120 s; a killed step is NOT MEASURED); the `order` check counted 0 of 234 per-app scripts (`apps//mdlsource/` now counted); and Stage 5 returned an empty verdict on any project with neither a look ledger nor a summary (`rows` unbound under `set -u`, introduced earlier in this PR) — parent session's field run - new(ci, ready): **the gate an agent cannot skip, and a READY nobody can hand-write.** `bin/ci-gates.sh` runs in the project's own CI (`.github/workflows/toolkit-gates.yml`, from `templates/ci/`, installed by `init-project.sh` and `sync-project.sh` create-if-absent, never overwritten, beside any existing CI) on every push and PR, over the committed tree at the pinned toolkit commit: the claimed stage's gate FAILs, a READY line `bin/ready.sh` did not write (`bin/lib/ready-claims.sh`, also a Stage 6 FAIL), committed unseen pages under a done-claim, and model work before Stage 4 all fail it (otherwise it prints `look: K of N pages seen` as a visible warning), in seconds, with no Mendix runtime. `ready.sh` now also requires the position (stages P and 0–4 PASS, WAIVED or not owed by the entry mode: a field run wrote READY with Stage P never passed), honours `Waived obligation must-haves`, writes a review packet (`docs/progress/review-packets/.html`: screenshot beside target with the reviewer's score, paths only), reads the must-have mark on its tracked path `docs/progress/must-haves/.md`, and names every app of a multi-app project instead of checking nothing (field runs, 2026-11: agents skip in-session gates; a hand-written READY graded a 0-of-78-looked build done) — two unattended field runs - new(ready, look-ledger, gate-check): **READY is earned through one command, and its evidence is in git.** `bin/ready.sh [--module M | --app A]` runs `gate-check.sh` Stage 5 (Stage 6 once the register is there), the look ledger, the must-have list when present and, under `Interview mode: auto`, an independent reviewer's verdict file (`docs/progress/look-reviews/`, every page scored 1–5 after its last build, below 4 blocks); it prints K of N per check, exits non-zero unless all pass, writes the READY line to `docs/progress/ready.log` itself and commits `docs/progress/ready--.md` with explicit paths. `look-ledger.sh summary --write` puts the git-ignored ledger into a tracked `docs/progress/look-summary.tsv` (written by `ready.sh` and every `close-task.sh`), and `gate-check.sh` Stage 5 reads it when the ledger is absent, so a copy of the repo reports FAIL instead of MANUAL. `dispatching.md` gains the reviewer dispatch rule; the `ready` routing row reaches `CLAUDE.local.md` and every agent stub (field runs, 2026-11: a build wrote READY by hand with 0 of ~78 pages looked at; another marked itself ready 3 hours early with no review pass) — two unattended field runs - fix(status, agents): **`bin/status.sh` WATCHes an Agent Editor project whose extension module does not travel with the model (#778)**: Studio Pro reads `agenteditor.agent` from `modules/AgentEditor.mxmodule` beside the `.mpr`, and a sync of only `.mpr` + `mprcontents/` left it out of git, so the builder stayed green while every colleague's clone showed `unknown type 'agenteditor.agent'`. Two WATCH items, warn-only (exit code and every verdict unchanged), local files and `git ls-files` only, root and `app/` layouts: a `modules/*.mxmodule` git does not track, and a project using Agent Editor (`AgentEditorCommons` in `analysis/app-facts/inventory.json`, or a `create agent` statement in `mdlsource/`) with no `modules/AgentEditor*.mxmodule`. `skills/mendix-agents.md` (on demand) gains the rule plus the note that the GenAI key, `Encryption.EncryptionKey` and `GenAICommons.StoreTraces` belong in the Default configuration, not a start script. Empty agent references / missing model version (mendixlabs/mxcli#1867) and unset runtime constants are not checked: they need a captured agent document (option 2). Fixture `tests/wave2/test-status-extension-modules.sh` — field report from an Agent Editor project, Nov 2026 diff --git a/bin/ci-gates.sh b/bin/ci-gates.sh index 3c9374d9..adc00054 100755 --- a/bin/ci-gates.sh +++ b/bin/ci-gates.sh @@ -114,18 +114,35 @@ EOF fi elif [ -n "$claims" ]; then say FAIL look "READY claimed ($(printf '%s' "$claims" | tr '\n' ';')) but no docs/progress/look-summary.tsv is committed: the look behind the claim is invisible" +elif [ -f "$ROOT/docs/BUILD-LOG.md" ] && grep -qE '^\| *[0-9]{4}-[0-9]{2}-[0-9]{2}' "$ROOT/docs/BUILD-LOG.md"; then + say WARN look "look: not measured — docs/BUILD-LOG.md has exec rows but no docs/progress/look-summary.tsv is committed (bin/ready.sh and close-task.sh write it)" else - say PASS look "no look summary committed and nothing claims done" + say PASS look "no look summary committed, no exec row, nothing claims done" fi # (d) model work before Stage 4 passed execs=0 [ -f "$ROOT/docs/BUILD-LOG.md" ] && execs="$(grep -cE '^\| *[0-9]{4}-[0-9]{2}-[0-9]{2}' "$ROOT/docs/BUILD-LOG.md" 2>/dev/null)" -scripts="$( [ -d "$ROOT/mdlsource" ] && find "$ROOT/mdlsource" -name '*.mdl' ! -path '*/gallery/*' 2>/dev/null | wc -l | tr -d ' ')" +# Script dirs: mdlsource/ at the root, plus one per app in a multi-app project (apps//mdlsource/, +# through the multi-app resolver mxtk_apps when this toolkit has it, else the apps/*/ glob). A field +# copy kept all 234 of its scripts per app, and the root-only count said 0. +dirs="$ROOT/mdlsource" +appdirs="$( ( PROJECT_ROOT="$ROOT"; . "$TOOLKIT_ROOT/project-bin/_common.sh" >/dev/null 2>&1 + type mxtk_apps >/dev/null 2>&1 && mxtk_apps "$ROOT" 2>/dev/null | cut -f2 ) < /dev/null 2>/dev/null )" +if [ -n "$appdirs" ]; then + for m in $appdirs; do case "$m" in /*) d="${m%/*}" ;; *) d="$ROOT/${m%/*}" ;; esac; dirs="$dirs $d/mdlsource"; done +else + for d in "$ROOT"/apps/*/mdlsource; do [ -d "$d" ] && dirs="$dirs $d"; done +fi +scripts=0 +for d in $dirs; do + [ -d "$d" ] || continue + scripts=$((scripts + $(find "$d" -name '*.mdl' ! -path '*/gallery/*' 2>/dev/null | wc -l | tr -d ' '))) +done if [ "${execs:-0}" -gt 0 ] || [ "${scripts:-0}" -gt 0 ]; then gate 4; rc=$? if [ "$rc" = 0 ]; then say PASS order "model work ($execs exec row(s), $scripts script(s)) after Stage 4: $GV" - else say FAIL order "model work ($execs exec row(s) in docs/BUILD-LOG.md, $scripts build script(s) in mdlsource/) while Stage 4 is ${GV:-not passed}: build before the plan. Pass Stage 4, or record the adoption point (gate-check.sh --adopt 5 --reason \"...\")"; fi + else say FAIL order "model work ($execs exec row(s) in docs/BUILD-LOG.md, $scripts build script(s) in mdlsource/ dirs) while Stage 4 is ${GV:-not passed}: build before the plan. Pass Stage 4, or record the adoption point (gate-check.sh --adopt 5 --reason \"...\")"; fi else say PASS order "no model work committed yet" fi diff --git a/bin/gate-check.sh b/bin/gate-check.sh index 582da7e2..45762a60 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -2215,7 +2215,7 @@ stage4_closing_rows() { # citations and waivers still clear them. MXTK_LOOK_MODULE= scopes the check to one module # (bin/ready.sh --module); unset, nothing changes. check_stage_5() { - local owed="$PROJECT_DIR/.claude/loop/look/owed.tsv" rows proofs="" n=0 n_seen=0 n_proof=0 n_waived=0 + local owed="$PROJECT_DIR/.claude/loop/look/owed.tsv" rows="" proofs="" n=0 n_seen=0 n_proof=0 n_waived=0 local unseen="" n_unseen=0 st page epoch mod rep line shot f sz mt src="" local summ="$PROJECT_DIR/docs/progress/look-summary.tsv" scope="${MXTK_LOOK_MODULE:-}" if [ -s "$owed" ] && [ -f "$TOOLKIT_DIR/project-bin/look-ledger.sh" ]; then diff --git a/bin/lib/ready-claims.sh b/bin/lib/ready-claims.sh index 91d3bafa..ea5109b5 100644 --- a/bin/lib/ready-claims.sh +++ b/bin/lib/ready-claims.sh @@ -6,8 +6,11 @@ # was graded done with 0 of ~78 pages looked at. bin/ready.sh is now the only writer of READY; this # is the check that a READY line anywhere else is backed by it. # -# A READY CLAIM is a line in a status file whose first word (after list, quote and bold markup, or -# a `Status:` / `State:` / `Verdict:` label) is READY, or a table cell that is exactly READY. +# A READY CLAIM is a line in a status file whose first word (after heading, list, quote and bold +# markup, or a `Status:` / `State:` / `Verdict:` label) is READY, whatever follows it, or a table cell +# that is exactly READY. Field shape (2026-11, five times in one status file, missed by the first +# version of this check, which only took READY alone or before `:`/`—`): +# # READY FOR BENCH (fix round 6, 2026-11-16 15:34 UTC) # Status files: PROJECT.md, RESUME.md / STATUS*.md / PROGRESS*.md at the root, and every .md under # docs/ except ready.sh's own output (docs/progress/ready-*.md, review-packets/, look-reviews/, # must-haves/). Not CLAUDE*.md, README or agent text: they explain the rule, they claim nothing. @@ -50,7 +53,7 @@ mxtk_ready_claim_faults() { out="$out$rel:$n: $(printf '%s' "$text" | cut -c1-100) — $why " done <*_#-]*(\*\*)?((Status|State|Verdict)[[:space:]]*:[[:space:]]*)?(\*\*)?READY(\*\*)?([[:space:]]*$|[[:space:]]*[:(]|[[:space:]]+[—–-]|[[:space:]]+(for|—)[[:space:]])|\|[[:space:]]*(\*\*)?READY(\*\*)?[[:space:]]*\|') +$(tr -d '\r' < "$f" | grep -nE '^[[:space:]>*_#-]*(\*\*)?((Status|State|Verdict)[[:space:]]*:[[:space:]]*)?(\*\*)?READY(\*\*)?([^A-Za-z0-9_-]|$)|\|[[:space:]]*(\*\*)?READY(\*\*)?[[:space:]]*\|') EOF done < "${TMPDIR:-/tmp}/mxtk-rc.$$" rm -f "${TMPDIR:-/tmp}/mxtk-rc.$$" diff --git a/bin/ready.sh b/bin/ready.sh index d301be7d..02ae7990 100755 --- a/bin/ready.sh +++ b/bin/ready.sh @@ -95,12 +95,38 @@ _epoch() { # ISO-8601 UTC -> epoch, GNU then BSD; 0 when unparseable date -u -d "$1" +%s 2>/dev/null || date -u -j -f '%Y-%m-%dT%H:%M:%SZ' "$1" +%s 2>/dev/null || echo 0 } +# step