diff --git a/CHANGELOG.md b/CHANGELOG.md index 96185415..3a697e5b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,10 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. Named m `docs/RELEASE-PROCESS.md` §8. ## Unreleased +- fix(ci, ready): **review of #788: the toolkit's own text no longer trips its READY check, CI no longer goes red mid-stage, and synced projects start advisory.** The `RESUME.md` starter and `close-task.sh` began a line with READY ("**READY is not written in this file.**"), so every fresh project failed CI and Stage 6 could never pass (now "Only bin/ready.sh writes READY"; a fixture runs init → commit → CI). Check (a) now fails only on stages the register records as passed and warns on the stage in progress (a fresh project at Stage P was red from its first push). `MXTK_CI_ENFORCE` in the workflow: `init-project.sh` installs "1", `sync-project.sh` installs "0" so stage and order checks warn until switched on; the hand-written READY check fails either way — independent review of #788 +- fix(ready, ci, gate-check): **field run of the CI gate and `ready.sh` on a six-app copy (#788)**: a hand-written `# READY FOR BENCH (fix round 6, …)` heading passed the `ready` check (READY followed by words now counts; heading markup stripped); `ready.sh` ran over 300 s with no output (every external step now names itself, has stdin closed and a watchdog, `MXTK_READY_STEP_TIMEOUT`, default 120 s; a killed step is NOT MEASURED); the `order` check counted 0 of 234 per-app scripts (`apps//mdlsource/` now counted); and Stage 5 returned an empty verdict on any project with neither a look ledger nor a summary (`rows` unbound under `set -u`, introduced earlier in this PR); on the re-run, claims in a lowercase root `status.md` were not scanned (root status/progress/resume files now matched in any case), and an annotated `- Interview mode: auto (…)` line, which `interview-mode.sh` resolves to steering, waived the independent review (the raw line's first word now decides whether the review is owed) — parent session's field run +- new(ci, ready): **the gate an agent cannot skip, and a READY nobody can hand-write.** `bin/ci-gates.sh` runs in the project's own CI (`.github/workflows/toolkit-gates.yml`, from `templates/ci/`, installed by `init-project.sh` and `sync-project.sh` create-if-absent, never overwritten, beside any existing CI) on every push and PR, over the committed tree at the pinned toolkit commit: the claimed stage's gate FAILs, a READY line `bin/ready.sh` did not write (`bin/lib/ready-claims.sh`, also a Stage 6 FAIL), committed unseen pages under a done-claim, and model work before Stage 4 all fail it (otherwise it prints `look: K of N pages seen` as a visible warning), in seconds, with no Mendix runtime. `ready.sh` now also requires the position (stages P and 0–4 PASS, WAIVED or not owed by the entry mode: a field run wrote READY with Stage P never passed), honours `Waived obligation must-haves`, writes a review packet (`docs/progress/review-packets/.html`: screenshot beside target with the reviewer's score, paths only), reads the must-have mark on its tracked path `docs/progress/must-haves/.md`, and names every app of a multi-app project instead of checking nothing (field runs, 2026-11: agents skip in-session gates; a hand-written READY graded a 0-of-78-looked build done) — two unattended field runs +- new(ready, look-ledger, gate-check): **READY is earned through one command, and its evidence is in git.** `bin/ready.sh [--module M | --app A]` runs `gate-check.sh` Stage 5 (Stage 6 once the register is there), the look ledger, the must-have list when present and, under `Interview mode: auto`, an independent reviewer's verdict file (`docs/progress/look-reviews/`, every page scored 1–5 after its last build, below 4 blocks); it prints K of N per check, exits non-zero unless all pass, writes the READY line to `docs/progress/ready.log` itself and commits `docs/progress/ready--.md` with explicit paths. `look-ledger.sh summary --write` puts the git-ignored ledger into a tracked `docs/progress/look-summary.tsv` (written by `ready.sh` and every `close-task.sh`), and `gate-check.sh` Stage 5 reads it when the ledger is absent, so a copy of the repo reports FAIL instead of MANUAL. `dispatching.md` gains the reviewer dispatch rule; the `ready` routing row reaches `CLAUDE.local.md` and every agent stub (field runs, 2026-11: a build wrote READY by hand with 0 of ~78 pages looked at; another marked itself ready 3 hours early with no review pass) — two unattended field runs - process(obligations): **`bin/lib/obligations.tsv` gains column 10 `modes`, the same column, tokens and tokeniser as `artifact-manifest.tsv` (`migration,requirements,greenfield,existing-app-change`, read through artifact-check.sh's own `_art_entry_mode`, never a second copy of its mapping)**: a row the register's `Entry mode:` does not list reads `N/A — not owed (entry mode …)`, never PASS; no or unrecognised mode = owed. Every row lists all four modes today, so no verdict changes; the column lets a mode that genuinely skips a pass be excused by data instead of by absence. Fixture `test-obligation-quality-floor.sh` T9 — follow-up to #787 - fix(obligations): **`Must-haves: advisory` for projects already past Stage 5 when the must-haves row arrived (follow-up to #787)**: `sync-project.sh` writes one `Must-haves:` line into `PROJECT.md` when none exists, and only then: `advisory — joined at stage N before the must-haves row existed` when gate-check's derived Current stage or `Adopted at stage:` is 6 or 7, else `bound`. `init-project.sh` writes `bound` for new projects. An advisory project's `Obligation must-haves` row reads `ADVISORY` and is left out of the pending count; `bound` keeps PENDING. It is keyed on the written line, not the project's position: a position check would silence a new project the day it passed Stage 5 without the marks. Fixture `tests/wave2/test-must-haves-advisory.sh` — owner request at the review of #787 - fix(obligations): **a quality floor in every entry mode: no per-module obligation reads PASS by absence, and a new `must-haves` row owes one mode-independent checklist per module**: with no module brief and no verify run, `obligation-check.sh` printed `Obligation look PASS — no module has been opened for work yet; nothing owed` (and the same for fidelity, sweep, journeys) on a project that had built ~78 pages and reviewed 0. Now an empty planned scope falls back to the modules that hold built pages, read from the look ledger (`.claude/loop/look/owed.tsv`), else page statements in the project's `*.mdl` scripts, else `.claude/loop/page-scope.json`, each looked for at the root, `app/` and `apps/*/`; every module row reads PENDING with an `OWED BY WHAT WAS BUILT: N page(s) built in M module(s) (...), K of M module(s) discharged` line. A model that has been built on (exec rows in `docs/BUILD-LOG.md`, `*.mdl` scripts, or a derived stage at 5+) with no countable pages is one FAULT naming `bin/page-scope.sh`, never "0 pages"; a `.mpr` with no build evidence (a fresh scaffold) keeps master's PASS line, so `bin/status.sh` still reads `modules opened: 0`. `existing-app-change` never takes scope from `page-scope.json`, which lists the whole model. A project with briefs keeps every earlier verdict. `skills/must-haves.md` (new, on demand) is the single source of poc-build.md's 12-row must-have checklist, generalised to every build type, plus row 13: every region the design shows is on the built page, counted; `poc-build.md` points to it, `module-review.md` stage 5 requires its mark `docs/progress/must-haves/.md` (first line `13 of 13 rows ticked or NOT DONE with a reason`). Informational, like every obligation row: no gate verdict or exit code changes; synced projects at Stage 5+ gain one `must-haves` PENDING row per module. Always-loaded words 64,984 -> 65,032 (budget 80,000). Fixture `tests/wave2/test-obligation-quality-floor.sh` — two field runs, unattended and requirements-driven, Nov 2026 diff --git a/README.md b/README.md index bd3d799c..1e22977a 100644 --- a/README.md +++ b/README.md @@ -915,6 +915,7 @@ Then filter by Role(s). A helper dispatched with a role stub (ba, architect, mdl | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | `bin/exec-approval.sh` | 5,6 | mdl, gate | | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | `skills/testing-shape.md` | 5,6 | test, gate, review | | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | `project-bin/verify-module.sh` | 5,6 | mdl, gate, test, review | +| Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | `bin/ready.sh` | 5,6 | every role | | A CSS/SCSS or style change that appears to have done nothing (a colour or class that won't show), a focus ring or outline that paints on every page landing or load, or an app still grey after a design port every instrument called green — the three ways a correct rule paints nothing (matches nothing / matches chrome / loses the cascade), the two reads that tell them apart, and the class that arrived in the stylesheet and is bound to no widget | `skills/learned-css-that-never-applied.md` | 5,6 | review, gate (mdl: on demand) | | Before trusting a green check/exec/DESCRIBE result as proof, or when a runtime symptom appears over a fully green model — the register of constructs that pass early rungs and fail later ones | `skills/learned-detection-gaps.md` | 5,6 | gate, review (mdl: on demand) | | Creating any entity, or calling a module security-ready — entity and grants land in one script, and ready means SHOW SECURITY MATRIX proves it | `skills/security-is-not-a-later-script.md` | 5 | gate, review (mdl: on demand) | diff --git a/ROUTING.md b/ROUTING.md index 8fabfa57..60a66f93 100644 --- a/ROUTING.md +++ b/ROUTING.md @@ -225,6 +225,7 @@ picks the row up. That is the whole procedure — there is no second list to rem |---|---|---|---|---| | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | `skills/testing-shape.md` | test,gate,review | 5,6 | baseline | | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | `project-bin/verify-module.sh` | mdl,gate,test,review | 5,6 | baseline | +| Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | `bin/ready.sh` | all | 5,6 | baseline | | Auditing or regression/e2e-testing an EXISTING app — no intake, no stages, no gates | `skills/existing-app-assurance.md` | test,review | 6 | ondemand | | Running the ledger rung alone — recompute every stored ledger status against the live model and catch the STALE rows that claim built for something the model no longer has | `project-bin/conformance-check.sh` | gate,review | 5,6 | ondemand | | Running the wiring rung alone — a module imported but never reached, an element built but wired to nothing, a boundary crossed; mxbuild and e2e are blind to all three | `project-bin/graph-sweep.sh` | review,gate | 5,6 | ondemand | diff --git a/agents/architect-agent.md b/agents/architect-agent.md index c357020c..2b6a66ed 100644 --- a/agents/architect-agent.md +++ b/agents/architect-agent.md @@ -41,6 +41,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/checkpoints/checkpoint-template.md` | Any stage transition — the 2+1 format every CAC uses, and the one-register rule (answers land in PROJECT.md, never in a separate state file). The seven CACs themselves are routed per stage in the situational table | | `skills/module-folder-convention.md` | Placing any document in a module — before the first `create`. Feature group, then Pages/Microflows/Services/Resources; the path comes from the brief's folder plan, and the table says which types mxcli can actually place | | `project-bin/check-design-reaches-app.sh` | After the FIRST build that follows any design-system port, and before any page is built on it — reads the BUILT stylesheet and reports how many framework knobs point at a design token, how many tokens arrived, how many component classes arrived, each with its denominator. Measured on a real run: 55 tokens ported correctly into the right file, 0 of 35 knobs bound and 0 of 20 classes present, two build phases shipped in the framework's default blue with mx check, mxcli lint, the MDL suite and two e2e journeys all green | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/status.sh` | The first command of every session, and any time someone asks "where are we" or "what next" — one screen: stage, done/overdue, the ONE next action, from the instruments, never from memory | | `skills/retesting-learned-rules.md` | Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | diff --git a/agents/ba-agent.md b/agents/ba-agent.md index 1b9d0d4b..7f3347bb 100644 --- a/agents/ba-agent.md +++ b/agents/ba-agent.md @@ -53,6 +53,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `bin/question-kinds.sh` | Deciding who answers a question — before putting any batch to the user. gap/conflict/choice/user-only is what keeps a gate batch at four questions instead of 127 | | `bin/facts-lock.sh` | Writing BRDs, especially several in parallel — "build" before the fan-out, "check" before any BRD is called done | | `skills/module-brief.md` | Building any module — before the first script. The mdl-agent's single per-module input | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/status.sh` | The first command of every session, and any time someone asks "where are we" or "what next" — one screen: stage, done/overdue, the ONE next action, from the instruments, never from memory | | `skills/retesting-learned-rules.md` | Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | diff --git a/agents/gate-agent.md b/agents/gate-agent.md index 2563531f..f222b096 100644 --- a/agents/gate-agent.md +++ b/agents/gate-agent.md @@ -40,6 +40,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `bin/exec-approval.sh` | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | | `skills/testing-shape.md` | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `skills/learned-css-that-never-applied.md` | A CSS/SCSS or style change that appears to have done nothing (a colour or class that won't show), a focus ring or outline that paints on every page landing or load, or an app still grey after a design port every instrument called green — the three ways a correct rule paints nothing (matches nothing / matches chrome / loses the cascade), the two reads that tell them apart, and the class that arrived in the stylesheet and is bound to no widget | | `skills/learned-detection-gaps.md` | Before trusting a green check/exec/DESCRIBE result as proof, or when a runtime symptom appears over a fully green model — the register of constructs that pass early rungs and fail later ones | diff --git a/agents/mdl-agent.md b/agents/mdl-agent.md index 7e04c7be..b71cc915 100644 --- a/agents/mdl-agent.md +++ b/agents/mdl-agent.md @@ -47,6 +47,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `project-bin/page-fidelity.js` | After drafting and again after exec'ing any page script — scores the page MDL (or `mxcli describe` output on stdin) against its wireframe: headings/actions/content/classes, weighted. The scored companion to check-page-shell's binary gate; 32% median measured without it, 90% first-draft with it. Every run is appended to the project's docs/PAGE-FIDELITY.tsv — first non-stub row per page = first-build score of record vs the ≥80% target (forward-reference stubs score with --stub, exempt) | | `bin/exec-approval.sh` | Before any mxcli exec / exec.sh / --mcp write — ask or run? the knob decides | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/bug-lookup.sh` | A CE error (mx check or mxbuild: CE0066, CE0117…), a cited BUG-n, or behavior that looks like a known mxcli quirk rather than a modeling mistake — `bin/bug-lookup.sh CE0117` / `BUG-102` / "keyword" / the whole pasted error message (ranked top 3) prints the matching ledger entries, so the session reads one entry, not the whole ledger | | `bin/file-bugs.sh` | You hit, or worked around, a bug in mxcli or the toolkit, or log a FINDINGS.md entry about either — `bin/file-bugs.sh` (the `--add` line is in learned-detection-gaps.md) keeps docs/bugs.jsonl, redacts, shows duplicates, files with `--auto` when PROJECT.md says `Bug filing: auto`, else only a human files (`!`, `--file PBUG-NN`, y/N, denied to agents), or prints new-issue links (`--links`, `--json`) | diff --git a/agents/review-agent.md b/agents/review-agent.md index d0e1487f..98e66e8f 100644 --- a/agents/review-agent.md +++ b/agents/review-agent.md @@ -47,6 +47,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/module-review.md` | Is the module done? Reviewing any module before calling it done — the ONE pass, the once-over: build, gate, prove, LOOK (is it logical, does it look right, does it match our design, over every page not just the tested ones), confirm with the denominator stated | | `skills/testing-shape.md` | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `skills/learned-css-that-never-applied.md` | A CSS/SCSS or style change that appears to have done nothing (a colour or class that won't show), a focus ring or outline that paints on every page landing or load, or an app still grey after a design port every instrument called green — the three ways a correct rule paints nothing (matches nothing / matches chrome / loses the cascade), the two reads that tell them apart, and the class that arrived in the stylesheet and is bound to no widget | | `skills/learned-detection-gaps.md` | Before trusting a green check/exec/DESCRIBE result as proof, or when a runtime symptom appears over a fully green model — the register of constructs that pass early rungs and fail later ones | diff --git a/agents/test-agent.md b/agents/test-agent.md index 4403acee..f8c5286d 100644 --- a/agents/test-agent.md +++ b/agents/test-agent.md @@ -39,6 +39,7 @@ This table is your whole list. The baseline table in the project's CLAUDE.local. | `skills/module-review.md` | Is the module done? Reviewing any module before calling it done — the ONE pass, the once-over: build, gate, prove, LOOK (is it logical, does it look right, does it match our design, over every page not just the tested ones), confirm with the denominator stated | | `skills/testing-shape.md` | Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature | | `project-bin/verify-module.sh` | Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh | +| `bin/ready.sh` | Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI | | `skills/tool-output-is-not-ground-truth.md` | Any time an exit code, a tool's output or a subagent's report is about to become a stated finding — verify before you conclude | | `bin/status.sh` | The first command of every session, and any time someone asks "where are we" or "what next" — one screen: stage, done/overdue, the ONE next action, from the instruments, never from memory | | `skills/retesting-learned-rules.md` | Before obeying any learned-* STOP or workaround that costs a detour — probe the binary you actually have, then stamp the verdict back into the rule | diff --git a/bin/ci-gates.sh b/bin/ci-gates.sh new file mode 100755 index 00000000..080d1072 --- /dev/null +++ b/bin/ci-gates.sh @@ -0,0 +1,177 @@ +#!/usr/bin/env bash +# ci-gates.sh — the gate an agent cannot skip. Run by the project's CI (.github/workflows/toolkit-gates.yml, +# installed by init-project.sh / sync-project.sh from templates/ci/toolkit-gates.yml) on every push and PR, +# over the COMMITTED tree only, with the toolkit checked out at the commit PROJECT.md pins. +# +# Usage: bin/ci-gates.sh exit 0 clean (warnings allowed), 1 a gate failed, 2 usage +# MXTK_CI_STRICT_LOOK=1 unseen built pages fail even with no done-claim (default: warn mid-build) +# MXTK_CI_ENFORCE=0 (a) and (d) warn instead of failing — what sync-project.sh installs into an +# existing project; init-project.sh installs "1". (b) is always enforced. +# +# WHY (field runs, 2026-11, unattended). Everything that runs inside the agent's session can be skipped +# by the agent: it can not run gate-check, commit with --no-verify, or write READY by hand — and in one +# run it did the last, with 0 of ~78 pages looked at. CI runs outside the session, on what was pushed. +# +# THE CHECKS +# (a) stage gate-check.sh for the stage the register's "## Current stage" line claims; FAIL fails. +# PENDING / MANUAL do not (a stage in progress has nothing to show yet). +# (b) ready a READY line in a status file that bin/ready.sh did not write (bin/lib/ready-claims.sh). +# (c) look the committed docs/progress/look-summary.tsv shows built pages never looked at, while +# something claims done: a READY row in docs/progress/ready.log covering them, or the +# register at Stage 6 or later. With no claim it is a warning (mid-build is normal and an +# always-red CI gets ignored); MXTK_CI_STRICT_LOOK=1 makes it fail. A READY with no +# committed summary behind it fails: the claim cites evidence nobody can read. +# (d) order model work before its plan: docs/BUILD-LOG.md has exec rows (or mdlsource/ holds build +# scripts outside gallery/) while gate-check Stage 4 is not PASS/WAIVED. gate-check already +# waives Stage 4 for greenfield and for an --adopt point at or after it. +# +# WHAT IT CANNOT SEE (anything not committed): the git-ignored .claude/loop/ ledgers themselves (only +# the summary ready.sh / close-task.sh commit), screenshots, chat, a model edited without exec.sh and +# without a BUILD-LOG row, and a summary nobody committed. It reads files, never the .mpr: no Mendix +# runtime, no mxcli. It runs in seconds (three gate-check stage queries at most). +# +# Producer for every consumer (CLAUDE.md "Shipping an instrument" rule 5): look-summary.tsv and ready.log +# are written by bin/ready.sh and close-task.sh; BUILD-LOG rows by exec.sh; the register by the gates. + +set -u +TOOLKIT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +ROOT="${1:-}" +[ -n "$ROOT" ] && [ -d "$ROOT" ] || { echo "usage: ci-gates.sh " >&2; exit 2; } +ROOT="$(cd "$ROOT" && pwd)" +REG="$ROOT/PROJECT.md"; GC="$TOOLKIT_ROOT/bin/gate-check.sh" +SUMM="$ROOT/docs/progress/look-summary.tsv"; LOG="$ROOT/docs/progress/ready.log" +FAILS=0; WARNS=0 +say() { # + case "$1" in FAIL) FAILS=$((FAILS+1)) ;; WARN) WARNS=$((WARNS+1)) ;; esac + printf '%-4s %-6s %s\n' "$1" "$2" "$3" + if [ -n "${GITHUB_ACTIONS:-}" ]; then + case "$1" in FAIL) echo "::error title=toolkit gate $2::$3" ;; WARN) echo "::warning title=toolkit gate $2::$3" ;; esac + fi +} +# gate — sets GV (verdict word) and GL (detail), returns gate-check's exit code +gate() { + local out rc + out="$(MXTK_NO_GUIDE=1 bash "$GC" --no-html "$ROOT" "$1" 2>&1)"; rc=$? + GL="$(printf '%s\n' "$out" | grep -m1 "^Stage $1 (" | sed "s/^Stage $1 ([^)]*): //")" + GV="$(printf '%s' "$GL" | awk '{print $1}')" + return $rc +} + +echo "ci-gates — $ROOT (toolkit $(git -C "$TOOLKIT_ROOT" rev-parse --short HEAD 2>/dev/null || echo '?'))" +[ -f "$REG" ] || { say FAIL stage "no PROJECT.md: not a toolkit project, or the register was never committed"; echo "FAIL — 1 gate"; exit 1; } + +# Enforcement (owner policy): a project that init-project.sh creates gets MXTK_CI_ENFORCE "1" in its +# workflow; one that sync-project.sh reaches mid-flight gets "0", so (a) and (d) start as warnings there +# until the project switches them on. (b) — a hand-written READY — is enforced everywhere, always. +# Unset (a run by hand) means enforced. +ENFORCE="${MXTK_CI_ENFORCE:-1}" +enf() { # — FAIL when enforced, else an advisory WARN that says how to turn it on + if [ "$ENFORCE" = 1 ]; then say FAIL "$1" "$2" + else say WARN "$1" "$2 (advisory: set MXTK_CI_ENFORCE: \"1\" in .github/workflows/toolkit-gates.yml to enforce)"; fi +} +[ "$ENFORCE" = 1 ] || echo "note enforcement is off for stage/order (MXTK_CI_ENFORCE=$ENFORCE): they warn; the READY check still fails" + +# (a) the stages the register records as passed must still pass. Only those: the stage in progress +# has its work still to do, and a fresh project at Stage P with intake unanswered would otherwise be +# red from its first push. The derived "## Current stage" line lists them ("gates passed: P, 0, 1"). +pos="$(awk '/^## Current stage/{f=1;next} f && /^\*\*/{print;exit}' "$REG")" +case "$pos" in + *"All gates passed"*) claim=""; passed="P 0 1 2 3 4 5 6 7" ;; + *) claim="$(printf '%s' "$pos" | grep -oE 'Stage [0-9P]' | head -1 | sed 's/^Stage //')" + passed="$(printf '%s' "$pos" | sed -n 's/.*gates passed: *\([^·(]*\).*/\1/p' | tr ',' ' ' | tr -cs 'P0-9' ' ')" ;; +esac +if [ -z "$claim" ] && [ -z "$passed" ]; then + say WARN stage "no \"## Current stage\" line with a stage in PROJECT.md — nothing claimed, nothing to check" +else + full="$(MXTK_NO_GUIDE=1 bash "$GC" --no-html "$ROOT" 2>&1 < /dev/null)" + verdict() { printf '%s\n' "$full" | grep -m1 "^Stage $1 (" | sed "s/^Stage $1 ([^)]*): //"; } + bad=0; np=0 + for st in $passed; do + np=$((np+1)); l="$(verdict "$st")" + case "$l" in FAIL*) bad=$((bad+1)); enf stage "Stage $st is recorded as passed in the register and its gate now FAILs: $(printf '%s' "$l" | cut -c1-220)" ;; esac + done + cur="" + if [ -n "$claim" ]; then + l="$(verdict "$claim")"; cur="; Stage $claim in progress: $(printf '%s' "$l" | awk '{print $1}')" + case "$l" in FAIL*) say WARN stage "Stage $claim is in progress and its gate FAILs (not a claim yet, so a warning): $(printf '%s' "$l" | cut -c1-200)" ;; esac + fi + [ "$bad" = 0 ] && say PASS stage "$np stage(s) recorded as passed still pass$cur" +fi + +# (b) READY written by hand +. "$TOOLKIT_ROOT/bin/lib/ready-claims.sh" +if rcf="$(mxtk_ready_claim_faults "$ROOT")"; then + say PASS ready "every READY line is backed by bin/ready.sh evidence" +else + while IFS= read -r l; do [ -n "$l" ] && say FAIL ready "$l"; done <= 4 { last[$3] = $2 } END { for (s in last) if (last[s] == "READY") print s }' "$LOG")" +late=0; case "$claim" in 6|7) late=1 ;; esac +case "$pos" in *"All gates passed"*) late=1; claim=7 ;; esac +if [ -s "$SUMM" ]; then + unseen="$(awk -F'\t' '$1 == "unseen" { print $2 }' "$SUMM")" + asof="$(awk -F'\t' '$1 == "as-of" { print $2 " (" $3 ", " $4 ")"; exit }' "$SUMM")" + # "look: K of N pages seen" — shown on every run that does not fail, so the number is in front of people + kn="$(awk -F'\t' '$1 == "as-of" { o=$3; s=$4; sub(/^owed /, "", o); sub(/^seen /, "", s); print "look: " s " of " o " pages seen"; exit }' "$SUMM")" + if [ -z "$unseen" ]; then + say PASS look "$kn — committed look summary, as of ${asof%% (*}" + else + n="$(printf '%s\n' "$unseen" | wc -l | tr -d ' ')" + hit="" + while IFS= read -r sc; do + [ -n "$sc" ] || continue + case "$sc" in + "module "*) m="${sc#module }"; printf '%s\n' "$unseen" | grep -q "^$m\." && hit="$hit${hit:+; }$sc" ;; + *) hit="$hit${hit:+; }$sc" ;; + esac + done </dev/null)" +# Script dirs: mdlsource/ at the root, plus one per app in a multi-app project (apps//mdlsource/, +# through the multi-app resolver mxtk_apps when this toolkit has it, else the apps/*/ glob). A field +# copy kept all 234 of its scripts per app, and the root-only count said 0. +dirs="$ROOT/mdlsource" +appdirs="$( ( PROJECT_ROOT="$ROOT"; . "$TOOLKIT_ROOT/project-bin/_common.sh" >/dev/null 2>&1 + type mxtk_apps >/dev/null 2>&1 && mxtk_apps "$ROOT" 2>/dev/null | cut -f2 ) < /dev/null 2>/dev/null )" +if [ -n "$appdirs" ]; then + for m in $appdirs; do case "$m" in /*) d="${m%/*}" ;; *) d="$ROOT/${m%/*}" ;; esac; dirs="$dirs $d/mdlsource"; done +else + for d in "$ROOT"/apps/*/mdlsource; do [ -d "$d" ] && dirs="$dirs $d"; done +fi +scripts=0 +for d in $dirs; do + [ -d "$d" ] || continue + scripts=$((scripts + $(find "$d" -name '*.mdl' ! -path '*/gallery/*' 2>/dev/null | wc -l | tr -d ' '))) +done +if [ "${execs:-0}" -gt 0 ] || [ "${scripts:-0}" -gt 0 ]; then + gate 4; rc=$? + if [ "$rc" = 0 ]; then say PASS order "model work ($execs exec row(s), $scripts script(s)) after Stage 4: $GV" + else enf order "model work ($execs exec row(s) in docs/BUILD-LOG.md, $scripts build script(s) in mdlsource/ dirs) while Stage 4 is ${GV:-not passed}: build before the plan. Pass Stage 4, or record the adoption point (gate-check.sh --adopt 5 --reason \"...\")"; fi +else + say PASS order "no model work committed yet" +fi + +if [ "$FAILS" -gt 0 ]; then echo "FAIL — $FAILS gate(s) failed, $WARNS warning(s)"; exit 1; fi +echo "PASS — $WARNS warning(s)" +exit 0 diff --git a/bin/gate-check.sh b/bin/gate-check.sh index 1065b9dd..e3a7c762 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -2207,15 +2207,29 @@ stage4_closing_rows() { # own "Marketplace installs:" line and never changes this verdict, so a project whose Stage 5 # exit is already recorded cannot flip to "approved over a failing gate". It blocks earlier, in # build-ready and in exec.sh before a domain-model script, where the install belongs. +# +# A checkout without the ledger (#T3, two field runs 2026-11): .claude/loop/ is git-ignored, so a +# copy of the repo has no owed.tsv and used to stay MANUAL over a build with 0 of ~78 pages looked +# at. docs/progress/look-summary.tsv is the TRACKED summary look-ledger.sh writes (bin/ready.sh, +# close-task.sh); its `unseen` rows are judged exactly like OWED ledger rows below, so PROOF-OF-LOOK +# citations and waivers still clear them. MXTK_LOOK_MODULE= scopes the check to one module +# (bin/ready.sh --module); unset, nothing changes. check_stage_5() { - local owed="$PROJECT_DIR/.claude/loop/look/owed.tsv" rows proofs="" n=0 n_seen=0 n_proof=0 n_waived=0 - local unseen="" n_unseen=0 st page epoch mod rep line shot f sz mt - if [ ! -s "$owed" ] || [ ! -f "$TOOLKIT_DIR/project-bin/look-ledger.sh" ]; then - check_stage_manual; return + local owed="$PROJECT_DIR/.claude/loop/look/owed.tsv" rows="" proofs="" n=0 n_seen=0 n_proof=0 n_waived=0 + local unseen="" n_unseen=0 st page epoch mod rep line shot f sz mt src="" + local summ="$PROJECT_DIR/docs/progress/look-summary.tsv" scope="${MXTK_LOOK_MODULE:-}" + if [ -s "$owed" ] && [ -f "$TOOLKIT_DIR/project-bin/look-ledger.sh" ]; then + # The toolkit's copy, not the project's: the same join for every project, whichever version + # of the script it installed. + rows="$(PROJECT_ROOT="$PROJECT_DIR" bash "$TOOLKIT_DIR/project-bin/look-ledger.sh" status 2>/dev/null)" + elif [ -s "$summ" ]; then + # SEEN rows are not listed by page in the summary: one placeholder per seen page of the scope. + rows="$(awk -F'\t' -v m="$scope" ' + $1=="module" && (m=="" || $2==m) { for (i=0; i<$4+0; i++) print "SEEN\t" $2 ".(seen)\t0" } + $1=="unseen" { print "OWED\t" $2 "\t" $3 }' "$summ")" + src=" (from the committed docs/progress/look-summary.tsv, $(awk -F'\t' '$1=="as-of"{print "as of " $2; exit}' "$summ"); the ledger is not in this checkout)" fi - # The toolkit's copy, not the project's: the same join for every project, whichever version - # of the script it installed. - rows="$(PROJECT_ROOT="$PROJECT_DIR" bash "$TOOLKIT_DIR/project-bin/look-ledger.sh" status 2>/dev/null)" + [ -n "$scope" ] && rows="$(printf '%s\n' "$rows" | awk -F'\t' -v m="$scope" 'index($2, m ".") == 1')" [ -n "$rows" ] || { check_stage_manual; return; } # PROOF-OF-LOOK citations, once: "Module.Pageshot-mtime" per citation with a real shot. @@ -2260,9 +2274,9 @@ EOF [ "$n_proof" -gt 0 ] && how="$how, $n_proof by PROOF-OF-LOOK" [ "$n_waived" -gt 0 ] && how="$how, $n_waived waived" if [ "$n_unseen" -gt 0 ]; then - echo "FAIL|look: $n_unseen of $n built page(s) never looked at since their last build: $unseen — screenshot each, open the PNG (a file name containing the page name, or the journey screenshot bin/look-ledger.sh shots lists), compare it to its wireframe (skills/ui-loop.md); or --waive look/ --reason \"...\"" + echo "FAIL|look: $n_unseen of $n built page(s)${scope:+ in $scope} never looked at since their last build$src: $unseen — screenshot each, open the PNG (a file name containing the page name, or the journey screenshot bin/look-ledger.sh shots lists), compare it to its wireframe (skills/ui-loop.md); or --waive look/ --reason \"...\"" else - echo "MANUAL|look: $n of $n built page(s) accounted for ($how); the rest of Stage 5 is manual" + echo "MANUAL|look: $n of $n built page(s)${scope:+ in $scope} accounted for ($how)$src; the rest of Stage 5 is manual" fi } @@ -2284,7 +2298,16 @@ test_surface_hit() { } check_stage_6() { - local test_ok="" review_ok="" + local test_ok="" review_ok="" rcf="" + # A READY line bin/ready.sh did not write is not READY (bin/lib/ready-claims.sh): Stage 6 FAILs on + # it before anything else, so a hand-written READY cannot carry a project into its test stage. + if [ -r "$TOOLKIT_DIR/bin/lib/ready-claims.sh" ]; then + . "$TOOLKIT_DIR/bin/lib/ready-claims.sh" + if ! rcf="$(mxtk_ready_claim_faults "$PROJECT_DIR")"; then + echo "FAIL|READY claimed without bin/ready.sh ($(printf '%s\n' "$rcf" | grep -c .) line(s)): $(printf '%s\n' "$rcf" | head -3 | tr '\n' ';' | sed 's/;$//') — READY is written only by bin/ready.sh [--module M]; remove the hand-written line or run it" + return + fi + fi test_surface_hit && test_ok=1 # module-review.md report — any non-empty dated report under a ui-reviews/ dir if [ -n "$(find_artifact -path '*/ui-reviews/ui-review-*.html' -size +0c)" ]; then diff --git a/bin/init-project.sh b/bin/init-project.sh index 6141025e..69fa24e9 100755 --- a/bin/init-project.sh +++ b/bin/init-project.sh @@ -291,6 +291,11 @@ else echo "Created: docs/progress/RESUME.md (starter — the first close-the-loop overwrites it)" fi +# .github/workflows/toolkit-gates.yml — the CI gate an agent cannot skip (bin/ci-gates.sh). Create-if- +# absent, never overwritten (bin/lib/ci-template.sh). +. "$TOOLKIT_ROOT/bin/lib/ci-template.sh" +mxtk_ci_install "$TOOLKIT_ROOT" "$PROJECT_DIR" 0 init || true + CLAUDE_LOCAL="$PROJECT_DIR/CLAUDE.local.md" if [ -f "$CLAUDE_LOCAL" ]; then echo "Skip: CLAUDE.local.md already exists — not overwritten." @@ -338,6 +343,8 @@ This project uses the shared toolkit at \`$TOOLKIT_ROOT\`. For ANY pipeline work 3. **Every gate question is asked in chat, then the turn ends and waits** (runbook §1). \`ASSUMED\` is earned by asking — the user said "you decide" — never by skipping. 4. **Before calling a stage done:** \`$TOOLKIT_ROOT/bin/gate-check.sh \`. + **READY / done on a module or the app only via** \`$TOOLKIT_ROOT/bin/ready.sh [--module M]\` + — it writes the READY line and commits the evidence; never write READY by hand. 5. **Before any page MDL:** read \`$TOOLKIT_ROOT/skills/ui-preflight-pages.md\` in full — its *text*; a delegated page draft carries the text or reads the file first. **After it:** the \`check-page-shell\`, \`page-fidelity\` and \`ui-loop\` rows below, then look at the page at diff --git a/bin/lib/ci-template.sh b/bin/lib/ci-template.sh new file mode 100644 index 00000000..6b3cd8cd --- /dev/null +++ b/bin/lib/ci-template.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +# ci-template.sh — install the project's CI gate (templates/ci/toolkit-gates.yml) as +# .github/workflows/toolkit-gates.yml. ONE copy, sourced by bin/init-project.sh and bin/sync-project.sh, +# for the same reason as ritual-template.sh: two hand-kept installers drift. +# +# Contract: CREATE-IF-ABSENT, never overwrite. The file is the project's once it exists (it may have +# been adapted: a fork's repo URL, a token, a runner); a copy that differs from the template is only +# reported. Other workflows in .github/workflows/ are left untouched: this is a separate workflow, +# not an edit of an existing one, so it sits beside whatever CI the repo already has. +# +# Sourced, never executed. Side-effect free until called. + +# mxtk_ci_install [dry-run 0|1] [init|sync] — prints one line (two +# for sync); returns 0 when it created the file (a change), 1 otherwise. +# init: the template as is, MXTK_CI_ENFORCE "1": a new project is bound from its first push. +# sync: MXTK_CI_ENFORCE "0": a project reached mid-flight gets warnings for the stage and order checks +# until it switches them on (owner policy, same softening as the must-have mark). The READY +# check is enforced in both. +mxtk_ci_install() { + local tk="$1" pr="$2" dry="${3:-0}" mode="${4:-init}" src dst other tmp + src="$tk/templates/ci/toolkit-gates.yml"; dst="$pr/.github/workflows/toolkit-gates.yml" + [ -f "$src" ] || return 1 + if [ -f "$dst" ]; then + if ! cmp -s "$src" "$dst" && ! sed 's/MXTK_CI_ENFORCE: "1"/MXTK_CI_ENFORCE: "0"/' "$src" | cmp -s - "$dst"; then + echo "Kept: .github/workflows/toolkit-gates.yml differs from the toolkit template (yours, not overwritten; compare with $src)" + fi + return 1 + fi + other="$(ls "$pr/.github/workflows" 2>/dev/null | grep -E '\.ya?ml$' | tr '\n' ' ')" + if [ "$dry" = 1 ]; then echo "Would create: .github/workflows/toolkit-gates.yml (the CI gate an agent cannot skip${mode:+, $mode})"; return 0; fi + mkdir -p "$pr/.github/workflows" || return 1 + if [ "$mode" = sync ]; then + sed 's/MXTK_CI_ENFORCE: "1"/MXTK_CI_ENFORCE: "0"/' "$src" > "$dst" || return 1 + else + cp "$src" "$dst" || return 1 + fi + echo "Created: .github/workflows/toolkit-gates.yml — the CI gate an agent cannot skip (bin/ci-gates.sh on every push and PR)${other:+; existing workflows left untouched: $other}" + [ "$mode" = sync ] && echo " Installed advisory: the stage and order checks warn until you set MXTK_CI_ENFORCE: \"1\" in that file; a hand-written READY fails the build already." + return 0 +} diff --git a/bin/lib/ready-claims.sh b/bin/lib/ready-claims.sh new file mode 100644 index 00000000..7483f568 --- /dev/null +++ b/bin/lib/ready-claims.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash +# ready-claims.sh — a READY line that bin/ready.sh did not write is not READY. ONE copy, sourced by +# bin/gate-check.sh (Stage 6) and bin/ci-gates.sh (the CI gate an agent cannot skip). +# +# WHY (field run, 2026-11, unattended). The build agent wrote READY into its own status file and +# was graded done with 0 of ~78 pages looked at. bin/ready.sh is now the only writer of READY; this +# is the check that a READY line anywhere else is backed by it. +# +# A READY CLAIM is a line in a status file whose first word (after heading, list, quote and bold +# markup, or a `Status:` / `State:` / `Verdict:` label) is READY, whatever follows it, or a table cell +# that is exactly READY. Field shape (2026-11, five times in one status file, missed by the first +# version of this check, which only took READY alone or before `:`/`—`): +# # READY FOR BENCH (fix round 6, 2026-11-16 15:34 UTC) +# Status files: PROJECT.md, and resume / status* / progress* .md at the root in any letter case, and every .md under +# docs/ except ready.sh's own output (docs/progress/ready-*.md, review-packets/, look-reviews/, +# must-haves/). Not CLAUDE*.md, README or agent text: they explain the rule, they claim nothing. +# BACKED means the line names an evidence file docs/progress/ready--.md that exists, +# whose verdict line (line 3, written by ready.sh) starts "READY:", and which docs/progress/ready.log +# lists as READY. Each scope's latest row in ready.log is held to the same test, so a hand-appended +# READY row with no evidence behind it is caught too; an older READY that a later run withdrew is not. +# +# What it cannot see: a claim made only in chat, a commit message, or an uncommitted file. A +# hand-forged evidence file passes this check; bin/ci-gates.sh then compares it with the committed +# look summary, which is where a forged claim and the ledger disagree. +# +# Sourced, never executed. Side-effect free. Bash 3.2. + +# mxtk_ready_claim_faults — one line per unbacked READY claim: +# :: — +# Returns 0 when there is none, 1 when there is at least one. +mxtk_ready_claim_faults() { + local root="$1" f rel log="$1/docs/progress/ready.log" out="" ev line n text why + _rc_backed() { # — 0 when ready.sh's READY evidence and its log row agree + [ -f "$root/$1" ] || return 1 + sed -n 3p "$root/$1" | grep -q '^READY:' || return 1 + [ -f "$log" ] && awk -F'\t' -v e="$1" '$2 == "READY" && $4 == e { f=1 } END { exit !f }' "$log" + } + { + # root status files in any case: a field copy kept five claims in a lowercase status.md + [ -f "$root/PROJECT.md" ] && printf '%s\n' "$root/PROJECT.md" + find "$root" -maxdepth 1 -type f \( -iname 'resume.md' -o -iname 'status*.md' -o -iname 'progress*.md' \) 2>/dev/null + [ -d "$root/docs" ] && find "$root/docs" -type f -name '*.md' \ + ! -path '*/docs/progress/ready-*' ! -path '*/docs/progress/review-packets/*' \ + ! -path '*/docs/progress/look-reviews/*' ! -path '*/docs/progress/must-haves/*' 2>/dev/null + } | sort -u > "${TMPDIR:-/tmp}/mxtk-rc.$$" + while IFS= read -r f; do + rel="${f#"$root"/}" + while IFS= read -r line; do + [ -n "$line" ] || continue + n="${line%%:*}"; text="${line#*:}" + ev="$(printf '%s' "$text" | grep -oE 'docs/progress/ready-[A-Za-z0-9_.-]+\.md' | head -1)" + if [ -z "$ev" ]; then why="no bin/ready.sh evidence named (docs/progress/ready--.md)" + elif ! _rc_backed "$ev"; then why="$ev is not a READY written by bin/ready.sh (missing, not READY, or not in docs/progress/ready.log)" + else continue; fi + out="$out$rel:$n: $(printf '%s' "$text" | cut -c1-100) — $why +" + done <*_#-]*(\*\*)?((Status|State|Verdict)[[:space:]]*:[[:space:]]*)?(\*\*)?READY(\*\*)?([^A-Za-z0-9_-]|$)|\|[[:space:]]*(\*\*)?READY(\*\*)?[[:space:]]*\|') +EOF + done < "${TMPDIR:-/tmp}/mxtk-rc.$$" + rm -f "${TMPDIR:-/tmp}/mxtk-rc.$$" + if [ -f "$log" ]; then + # Only each scope's LATEST row is a standing claim: a later NOT-READY run withdraws an earlier + # READY (and overwrites that day's evidence file), so judging old rows would fault an honest log. + while IFS=' ' read -r _iso v sc ev _k; do + [ "$v" = READY ] || continue + _rc_backed "$ev" || out="${out}docs/progress/ready.log: READY $sc — $ev is missing or does not say READY +" + done <= 4 { last[$3] = $0; if (!($3 in o)) ord[++k] = $3; o[$3] = 1 } END { for (i=1; i<=k; i++) print last[ord[i]] }' "$log") +EOF + fi + [ -z "$out" ] && return 0 + printf '%s' "$out" + return 1 +} diff --git a/bin/lib/resume-template.sh b/bin/lib/resume-template.sh index 1a079f60..bdef30ca 100644 --- a/bin/lib/resume-template.sh +++ b/bin/lib/resume-template.sh @@ -50,6 +50,9 @@ it). The first session to finish work overwrites it wholesale per confidently wrong on any project that was already mid-flight when this file appeared, and a wrong resume doc is worse than none. Step 1 below is how to find out. +**Only \`$toolkit/bin/ready.sh\` writes READY** (\`docs/progress/ready.log\` plus its evidence), +never this file; a READY line it did not write FAILs Stage 6 and CI. + ## Next actions, in order 1. Run \`$toolkit/bin/gate-check.sh .\` — it reports the current stage from the files on disk. diff --git a/bin/lib/skill-routing.tsv b/bin/lib/skill-routing.tsv index ce88bef5..32feb44a 100644 --- a/bin/lib/skill-routing.tsv +++ b/bin/lib/skill-routing.tsv @@ -124,6 +124,7 @@ exec-approval bin/exec-approval.sh Before any mxcli exec / exec.sh / --mcp write context-pack project-bin/context-pack.sh Dispatching a build step to a helper — generate its one-file pack (the brief's Build steps row, mxcli brain brief, live DESCRIBEs, example, folder) and hand over the path instead of a reading list. Measured: 25-38% fewer tool calls and 30-41% less time, same quality; the token saving is not proven mdl 5 ondemand build testing-shape skills/testing-shape.md Before calling any module tested, or before re-running tests after a change — what testing a module means, which re-runs a change owes (UI-only vs logic lane), and the false-green register of confirmed ways a test reports green over a broken feature test,gate,review 5,6 baseline verify verify-module project-bin/verify-module.sh Finishing any module — before calling it done. One command that runs every instrument and keeps "instrument faulted" apart from "feature failed"; in a wired project run the installed copy at bin/verify-module.sh mdl,gate,test,review 5,6 baseline verify +ready bin/ready.sh Claiming READY or done for a module or the app — the only way to write READY: `bin/ready.sh [--module M]` runs the Stage 5/6 gates, the look ledger, the must-have list and (unattended) the independent review, prints K of N for each and commits the evidence; a READY line it did not write is not READY and FAILs Stage 6 and CI all 5,6 baseline verify security-check project-bin/security-check.sh Before hand-over, before calling a module security-ready, or after any change to roles, access rules or project security — security lint findings and the who-can-see-what snapshot, each compared with what a person accepted; exits 3 on a first run, never 0; a person runs --approve/--accept mdl,gate,test,review 5,6 ondemand build/security mxcli-bugs bug-logs/mxcli-bugs.md Reading a whole class of tool defects (a retest, a new mxcli release, an audit) — for one CE code or symptom use bin/bug-lookup.sh instead; the ledger is too large to read whole mdl,gate 5,6 ondemand diagnose bootstrap-project skills/bootstrap-project.md Generating a new project's CLAUDE.md — baseline routing plus project-specific facts ba P ondemand spine diff --git a/bin/ready.sh b/bin/ready.sh new file mode 100755 index 00000000..f5bf8785 --- /dev/null +++ b/bin/ready.sh @@ -0,0 +1,433 @@ +#!/usr/bin/env bash +# ready.sh — the ONE way a module or an app is called READY / done. It runs the checks, prints +# K of N for each, writes the READY line itself and commits the evidence behind it. +# +# Usage: bin/ready.sh [--module | --app ] [--no-commit] +# --module M scope every check to module M (its pages, its must-have list, its review) +# --app A the whole app, labelled A (default: the whole project, labelled "app") +# --no-commit write the files, leave committing to the caller +# Exit 0 READY, 1 NOT READY, 2 usage. Bash 3.2 compatible. +# +# WHY (two field runs, 2026-11, both unattended). Run 1: the build agent wrote READY into its own +# status file and was graded done with 0 of ~78 built pages ever looked at. The look ledger was +# installed and active, and gate-check.sh's Stage 5 check would have FAILed the claim — but +# nobody ran it before claiming, and .claude/loop/ is git-ignored, so the orchestrator and the +# reviewers looking at the repo saw nothing (gate-check on a copy said MANUAL). Every look that +# did happen was the builder judging its own screens. Run 2, another kit, marked itself ready +# 3 hours before its time cap with no styling or review pass. A READY anyone may write by hand +# is a claim; this script makes it a verdict, and puts the evidence in git. +# +# THE CHECKS (each row prints K of N; a row nobody measured is NOT MEASURED and fails — never +# green-by-absence): +# position stages P and 0-4 each PASS, WAIVED (an --adopt point) or not owed by the entry mode. +# gate 5 gate-check.sh 5. PASS/WAIVED, or MANUAL with the look accounted for (Stage 5 +# is MANUAL by design once the look is clear — its other half is a Decisions row). +# gate 6 only when the register's "## Current stage" line is at Stage 6 or later. +# look pages owed a LOOK vs seen, per gate-check's own join (screenshots opened, +# PROOF-OF-LOOK citations, waivers — evidence this script did not create, guard +# rule 6). No page ever recorded as built and no `Waived obligation look` line: +# NOT MEASURED, a fail. +# must-have the per-module must-have list, when the project has one: K ticked of N. Absent: +# reported "not present", not counted. +# review only under `Interview mode: auto` (bin/interview-mode.sh): a verdict file written +# by a DIFFERENT agent (skills/dispatching.md "Independent look before READY"), every page of +# the scope scored 1-5 after its last build; a page below 4, or unscored, blocks. +# +# WHAT IT WRITES (all tracked, all small, no images): +# docs/progress/ready--.md the evidence: verdict, each row, owed vs seen per +# module, unseen pages, the screenshot names relied on +# docs/progress/ready.log one line per run: READY|NOT-READY +# — the READY line. An agent never writes it by hand. +# docs/progress/review-packets/.html per module: each page's screenshot beside its target with +# the reviewer's score, under the K-of-N lines. Paths only, no +# images: the page a person opens after an unattended run +# docs/progress/look-summary.tsv the git-ignored ledger's summary (look-ledger.sh summary) +# then `git commit -- ` (explicit paths: other sessions share the index). +# +# Producer for every consumer (CLAUDE.md "Shipping an instrument" rule 5): owed.tsv is written by +# exec.sh (the mandatory write path), seen.tsv by the Read hook install-claude-permissions.sh +# installs, the review file by the reviewer the lead dispatches (module-review.md), the must-have +# list by its own skill. ready.sh blocks nothing but the claim; it never blocks a write or a fix +# (guard rule 7): run it again after the fix. + +set -u +TOOLKIT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" + +usage() { echo "usage: ready.sh [--module | --app ] [--no-commit]" >&2; exit 2; } + +ROOT=""; MOD=""; APP=""; COMMIT=1 +while [ $# -gt 0 ]; do + case "$1" in + --module) shift; MOD="${1:-}"; [ -n "$MOD" ] || usage ;; + --app) shift; APP="${1:-}"; [ -n "$APP" ] || usage ;; + --no-commit) COMMIT=0 ;; + -h|--help) usage ;; + -*) echo "ready.sh: unknown option $1" >&2; usage ;; + *) [ -z "$ROOT" ] && ROOT="$1" || usage ;; + esac + shift +done +[ -n "$ROOT" ] && [ -d "$ROOT" ] || usage +[ -n "$MOD" ] && [ -n "$APP" ] && { echo "ready.sh: --module and --app are exclusive" >&2; usage; } +ROOT="$(cd "$ROOT" && pwd)" +REG="$ROOT/PROJECT.md" +if [ -n "$MOD" ]; then SCOPE="module $MOD"; elif [ -n "$APP" ]; then SCOPE="app $APP"; else SCOPE="app"; fi +SLUG="$(printf '%s' "${MOD:-${APP:-app}}" | tr -c 'A-Za-z0-9_-' '-' | sed 's/-*$//')" +DATE="$(date -u +%Y-%m-%d)"; ISO="$(date -u +%Y-%m-%dT%H:%M:%SZ)" +EVID_REL="docs/progress/ready-$DATE-$SLUG.md"; LOG_REL="docs/progress/ready.log" +LEDGER="$TOOLKIT_ROOT/project-bin/look-ledger.sh" +TMP="$(mktemp -d "${TMPDIR:-/tmp}/ready.XXXXXX")" || exit 2 +trap 'rm -rf "$TMP"' EXIT + +N_ROWS=0; N_OK=0 +: > "$TMP/rows.md" +# row — prints and records one check row +row() { + local mark + case "$1" in ok) mark="PASS"; N_OK=$((N_OK+1)); N_ROWS=$((N_ROWS+1)) ;; + fail) mark="FAIL"; N_ROWS=$((N_ROWS+1)) ;; + *) mark="n/a" ;; esac + printf ' %-4s %-9s %s\n' "$mark" "$2" "$3" + printf '| %s | %s | %s |\n' "$2" "$mark" "$(printf '%s' "$3" | sed 's/|/\\|/g')" >> "$TMP/rows.md" +} + +_epoch() { # ISO-8601 UTC -> epoch, GNU then BSD; 0 when unparseable + date -u -d "$1" +%s 2>/dev/null || date -u -j -f '%Y-%m-%dT%H:%M:%SZ' "$1" +%s 2>/dev/null || echo 0 +} + +# step \""
" esc(rel(x)) "
" + return "" esc(rel(x)) "" } + BEGIN { + while ((getline l < shotf) > 0) { split(l, a, "\t"); shot[a[2]] = a[3] } + while ((getline l < scf) > 0) { split(l, a, "\t"); if (!(a[1] in rat) || a[3]+0 > rat[a[1]]) { rat[a[1]]=a[3]+0; rsc[a[1]]=a[2]; rss[a[1]]=a[5]; rtg[a[1]]=a[6]; rnt[a[1]]=a[7] } } + print "" + print "Review packet" + print "

Review packet: " esc(scope) "

" esc(verdict) " · " esc(iso) " · written by bin/ready.sh. Images are not committed: open this file in the build checkout to see them.

" + print "" + while ((getline l < rowsf) > 0) { n = split(l, c, " \\| "); sub(/^\| /, "", c[1]); sub(/ \|$/, "", c[n]) + print "" } + print "
CheckVerdictDetail
" esc(c[1]) "" esc(c[2]) "" esc(c[3]) "
" + } + { m = $2; sub(/\..*/, "", m); if (!(m in seenm)) { ord[++k] = m; seenm[m] = 1 }; pages[m] = pages[m] SUBSEP $2; st[$2] = $1; built[$2] = $3+0 } + END { + if (k == 0) print "

No page recorded as built in this checkout: nothing to show.

" + for (i = 1; i <= k; i++) { + m = ord[i]; np = split(substr(pages[m], 2), pl, SUBSEP); ns = 0; nr = 0 + for (j = 1; j <= np; j++) { p = pl[j]; if (st[p] == "SEEN") ns++; if ((p in rat) && rat[p] >= built[p] && rsc[p]+0 >= 4) nr++ } + print "

" esc(m) "

" ns " of " np " page(s) seen since their last build · " nr " of " np " scored 4+ by an independent reviewer

" + print "" + for (j = 1; j <= np; j++) { p = pl[j] + sh = (p in rss && rss[p] != "") ? root "/" rss[p] : shot[p] + stale = ((p in rat) && rat[p] < built[p]) ? " (before last build)" : "" + sc = (p in rsc) ? rsc[p] stale : "unscored" + cls = (!(p in rsc) || stale != "" || rsc[p]+0 < 4) ? "low" : "" + print "" } + print "
PageScreenshotTargetScoreNote
" esc(p) "
" (st[p] == "SEEN" ? "seen" : "not seen") "
" media(sh) "" media((p in rtg && rtg[p] != "") ? root "/" rtg[p] : "") "" esc(sc) "" esc(rnt[p]) "
" + } + print "" + }' "$TMP/st" > "$ROOT/$PACKET_REL" +printf '%s\t%s\t%s\t%s\t%s of %s\n' "$ISO" "$VERDICT" "$SCOPE" "$EVID_REL" "$N_OK" "$N_ROWS" >> "$ROOT/$LOG_REL" +echo "evidence: $EVID_REL · packet: $PACKET_REL · log: $LOG_REL" + +# --- commit the evidence (explicit paths only) -------------------------------------------------- +if [ "$COMMIT" = 1 ] && git -C "$ROOT" rev-parse --git-dir >/dev/null 2>&1; then + paths="$EVID_REL $LOG_REL $PACKET_REL" + [ -f "$ROOT/docs/progress/look-summary.tsv" ] && paths="$paths docs/progress/look-summary.tsv" + # shellcheck disable=SC2086 + if ( cd "$ROOT" && git add -- $paths && git commit -q -m "ready: $SCOPE $VERDICT ($N_OK of $N_ROWS checks)" -- $paths ) >/dev/null 2>&1; then + echo "committed: $(git -C "$ROOT" log -1 --format='%h %s')" + else + echo " note the evidence was written but not committed (git refused) — commit these paths: $paths" >&2 + fi +fi + +[ "$VERDICT" = READY ] diff --git a/bin/sync-project.sh b/bin/sync-project.sh index d725f735..f2cba8ca 100755 --- a/bin/sync-project.sh +++ b/bin/sync-project.sh @@ -1367,6 +1367,14 @@ if [ "$WIRED" -eq 1 ] && [ ! -f "$PROJECT_DIR/docs/progress/RESUME.md" ]; then CHANGES=$((CHANGES + 1)) fi +# --- 4a1. CI gate: .github/workflows/toolkit-gates.yml, create-if-absent ------------------------ +# The gate an agent cannot skip (bin/ci-gates.sh, on every push and PR). Never overwritten: a copy +# that differs from templates/ci/toolkit-gates.yml is reported, not replaced (bin/lib/ci-template.sh). +if [ "$WIRED" -eq 1 ]; then + . "$TOOLKIT_ROOT/bin/lib/ci-template.sh" + mxtk_ci_install "$TOOLKIT_ROOT" "$PROJECT_DIR" "$DRY_RUN" sync && CHANGES=$((CHANGES + 1)) +fi + # --- 4b. Verification engine: tests/e2e/ (same contract as the crash net) ---------------- # project-bin/verify-module.sh has been installed by section 4 for weeks and it DRIVES # project-tests/e2e/journey-runner.js — which reached exactly one project, by hand, and that diff --git a/project-bin/close-task.sh b/project-bin/close-task.sh index 480a97c6..8eb9b3aa 100755 --- a/project-bin/close-task.sh +++ b/project-bin/close-task.sh @@ -119,6 +119,16 @@ if git rev-parse --git-dir >/dev/null 2>&1; then fi fi +# ---- the look ledger, as a tracked summary ------------------------------ +# .claude/loop/ is git-ignored, so without this "N owed, 0 seen" exists only in this container. +# look-ledger.sh rewrites docs/progress/look-summary.tsv (tracked) only when a count moved; the +# `git commit` below carries it. Fail-open: a missing ledger script prints nothing, owes nothing. +LOOK_SCRIPT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/look-ledger.sh" +if [ -f "$LOOK_SCRIPT" ] && [ -s "$ROOT/.claude/loop/look/owed.tsv" ]; then + look_line=$(PROJECT_ROOT="$ROOT" bash "$LOOK_SCRIPT" summary --write 2>/dev/null) + [ -n "$look_line" ] && { echo "### Look ledger"; echo; echo "- $look_line — commit docs/progress/look-summary.tsv with the rest"; echo "- Only \`$MXTK_ROOT/bin/ready.sh $ROOT [--module M]\` claims READY / done"; echo; } +fi + # ---- the routing table ------------------------------------------------- cat <<'TABLE' ### Where each thing goes — route by lifespan and audience, not topic diff --git a/project-bin/look-ledger.sh b/project-bin/look-ledger.sh index ef58751b..f0738619 100755 --- a/project-bin/look-ledger.sh +++ b/project-bin/look-ledger.sh @@ -3,11 +3,14 @@ # # bin/look-ledger.sh owe called by bin/exec.sh after a script lands # bin/look-ledger.sh seen PostToolUse(Read) hook; reads the hook JSON on stdin -# bin/look-ledger.sh status prints one line per owed page: SEEN or OWED +# bin/look-ledger.sh status [--shots] prints one line per owed page: SEEN or OWED (--shots: state, page, +# and the screenshot that saw it — bin/ready.sh's evidence list) # bin/look-ledger.sh reshoot [Module] [--all] # which pages a polish round must re-shoot (390 + 1280): the pages # built after their last seen LOOK, plus every page when a layout, # snippet or style file changed after it. K of N, never silence +# bin/look-ledger.sh summary [--write] pages owed vs opened, per module, plus each unseen page; --write +# also writes it to docs/progress/look-summary.tsv (TRACKED — see below) # bin/look-ledger.sh shots [findings.json...] # which owed page a journey run's screenshots already show: # OPEN (read it), UNREACHED (no journey did) @@ -361,13 +364,57 @@ RESHOOT_ROWS2 return 0 } +# cmd_summary [--write] — the ledger as a small TRACKED file, so "N owed, 0 seen" is visible from git. +# WHY (two field runs, 2026-11). .claude/loop/ is git-ignored, so the ledger lived only in the build's +# container: the orchestrator and the reviewers looking at the repo saw nothing, and gate-check.sh on a +# copy of the repo reported Stage 5 MANUAL for a build with 0 of ~78 pages ever looked at. The ledger +# itself stays ignored (it is append-only machine state, rewritten on every Read); this summary sits in +# docs/progress/ beside RESUME.md, which is tracked, instead of beside the ledger — a file inside an +# ignored directory cannot be re-included by a .gitignore negation. Producers: bin/ready.sh (every run) +# and close-task.sh (every work boundary, before its `git commit`). Consumer: gate-check.sh Stage 5, +# when the ledger is absent from the checkout. "seen" here is the raw ledger (a screenshot opened); +# gate-check also credits PROOF-OF-LOOK citations and waivers, so it reads the unseen rows, not the count. +# as-of owed seen unseen +# module +# unseen +SUMMARY_REL="docs/progress/look-summary.tsv" +_summary_body() { awk -F'\t' -v OFS='\t' '$1=="as-of" { $2="" } { print }'; } +cmd_summary() { + local rows out + rows="$(look_status)" + out="$(printf '%s\n' "$rows" | awk -F'\t' -v iso="$(date -u +%Y-%m-%dT%H:%M:%SZ)" ' + NF>=3 { m=$2; sub(/\..*/, "", m); if (!(m in o)) ord[++k]=m; o[m]++; n++ + if ($1=="SEEN") { s[m]++; ns++ } else un[++u]=$2 "\t" $3 } + END { + printf "# look-summary.tsv — written by look-ledger.sh summary from the git-ignored .claude/loop/look/ ledger; do not edit\n" + printf "as-of\t%s\towed %d\tseen %d\tunseen %d\n", iso, n, ns, n-ns + for (i=1; i<=k; i++) printf "module\t%s\t%d\t%d\n", ord[i], o[ord[i]], s[ord[i]]+0 + for (i=1; i<=u; i++) printf "unseen\t%s\n", un[i] + }')" + if [ "${1:-}" = --write ]; then + if [ -s "$OWED" ]; then + # rewritten only when a count or an unseen row moved: a new as-of alone is not worth a diff + if [ "$(_summary_body < "$PROJECT_ROOT/$SUMMARY_REL" 2>/dev/null)" != "$(printf '%s\n' "$out" | _summary_body)" ]; then + mkdir -p "$PROJECT_ROOT/docs/progress" 2>/dev/null && printf '%s\n' "$out" > "$PROJECT_ROOT/$SUMMARY_REL" 2>/dev/null + fi + fi + fi + if [ -s "$OWED" ]; then + printf '%s\n' "$out" | awk -F'\t' '$1=="as-of" { print "look ledger: " $3 ", " $4 ", " $5 " (as of " $2 ")" }' + else + echo "look ledger: NOT MEASURED — no $OWED (no page built through exec.sh in this checkout)" + fi + return 0 +} + case "${1:-}" in + summary) shift; cmd_summary "$@" ;; reshoot) shift; cmd_reshoot "$@" ;; owe) shift; cmd_owe "$@" ;; seen) cmd_seen ;; - status) look_status ;; + status) if [ "${2:-}" = --shots ]; then _look_status_ex | cut -f1,2,5; else look_status; fi ;; shots) shift; cmd_shots "$@" ;; pages) shift; look_pages_in "${1:-/dev/null}" ;; - *) echo "usage: $0 owe | seen < hook-json | status | reshoot [Module] [--all] | shots [findings.json...] | pages " >&2; exit 2 ;; + *) echo "usage: $0 owe | seen < hook-json | status [--shots] | summary [--write] | reshoot [Module] [--all] | shots [findings.json...] | pages " >&2; exit 2 ;; esac exit 0 diff --git a/skills/dispatching.md b/skills/dispatching.md index 50f77366..ba613141 100644 --- a/skills/dispatching.md +++ b/skills/dispatching.md @@ -104,6 +104,15 @@ every subagent: disable the ones a project does not use. The toolkit keeps its o (`CLAUDE.local.md`'s session-start ritual is main-session only and does not repeat what `gate-check.sh` prints). +**Independent look before READY (unattended).** Under `Interview mode: auto`, `bin/ready.sh` +refuses READY without `docs/progress/look-reviews/look-review-.md` from an agent that did +not build the pages. Dispatch a fresh reviewer, never the builder resumed. Its brief carries only +pairs: each page's screenshot and its target (wireframe, or the mapped prototype screen), never the +builder's verdict, notes or fidelity score. It writes `Reviewer:` and `Builder:` (different names), +`Reviewed: `, then a table `| Page | Screenshot | Target | Score | Note |`, one row +per page, Score 1–5 against the target. A page below 4, or rebuilt after its review, blocks READY. +Failure this prevents (field run, 2026-11): every look was the builder judging its own screens. + **Done when**, for each dispatch batch: N of N calls name a model; 0 scans started as forks; every fix round names its review file and commit range; 0 messages sent to agents that already reported. After a run, `bin/context-audit.sh --project ` section 1 gives the subagent start diff --git a/templates/ci/toolkit-gates.yml b/templates/ci/toolkit-gates.yml new file mode 100644 index 00000000..fc9a0aad --- /dev/null +++ b/templates/ci/toolkit-gates.yml @@ -0,0 +1,50 @@ +# toolkit-gates.yml — installed by mxcli-project-toolkit (bin/init-project.sh, bin/sync-project.sh). +# Never overwritten once present: edit it freely; sync only reports when it differs from the template. +# +# The gate an agent cannot skip. Everything inside an agent's session can be skipped by the agent +# (not running a gate, `git commit --no-verify`, writing READY by hand). This runs on GitHub, on what +# was pushed, and runs the toolkit's bin/ci-gates.sh at the toolkit commit PROJECT.md pins: +# (a) a stage the register records as passed whose gate now FAILs fails (the stage in progress warns) +# (b) a READY line that bin/ready.sh did not write fails +# (c) built pages never looked at fail once anything claims done (READY, or Stage 6+) +# (d) model work committed before Stage 4 passed fails (greenfield and --adopt points excepted) +# +# WHAT IT CANNOT SEE — anything not committed: the git-ignored .claude/loop/ ledgers (only the summary +# docs/progress/look-summary.tsv that bin/ready.sh and close-task.sh commit), screenshots, chat, and a +# model changed without exec.sh and without a docs/BUILD-LOG.md row. No Mendix runtime, no mxcli: it +# reads files only, in seconds. +# +# The toolkit repo defaults to the public one; set the repository variable MXTK_TOOLKIT_REPO to use a +# fork (a private fork also needs a read token in the checkout step). +name: toolkit-gates +on: + push: + pull_request: +permissions: + contents: read +jobs: + gates: + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - uses: actions/checkout@v4 + - name: Toolkit at the commit PROJECT.md pins + env: + TOOLKIT_REPO: ${{ vars.MXTK_TOOLKIT_REPO || 'https://github.com/MendixMau/mxcli-project-toolkit.git' }} + run: | + sha="$(sed -n 's/^Toolkit commit:[[:space:]`]*\([0-9a-f]\{7,40\}\).*/\1/p' PROJECT.md | head -1)" + git clone -q --filter=blob:none "$TOOLKIT_REPO" "$RUNNER_TEMP/toolkit" + if [ -n "$sha" ] && git -C "$RUNNER_TEMP/toolkit" checkout -q "$sha" 2>/dev/null \ + && [ -f "$RUNNER_TEMP/toolkit/bin/ci-gates.sh" ]; then + echo "toolkit pinned at $sha" + else + git -C "$RUNNER_TEMP/toolkit" checkout -q master + echo "::warning title=toolkit gates::PROJECT.md pins no usable toolkit commit (or one older than bin/ci-gates.sh); ran master instead" + fi + - name: Toolkit gates + env: + # "1": the stage (a) and order (d) checks fail the build. A project the toolkit reached + # mid-flight (sync-project.sh) starts at "0", where they only warn; set "1" when ready. + # The hand-written READY check (b) fails the build either way. + MXTK_CI_ENFORCE: "1" + run: bash "$RUNNER_TEMP/toolkit/bin/ci-gates.sh" . diff --git a/tests/wave2/test-ci-gates.sh b/tests/wave2/test-ci-gates.sh new file mode 100755 index 00000000..0136c662 --- /dev/null +++ b/tests/wave2/test-ci-gates.sh @@ -0,0 +1,186 @@ +#!/usr/bin/env bash +# test-ci-gates.sh — bin/ci-gates.sh, the CI gate an agent cannot skip, run the way CI runs it: on a +# fresh clone of the committed tree, without the git-ignored .claude/loop/ ledgers. One project tree per +# failure case, plus a clean one and the installer's never-overwrite contract. +# +# Golden input is produced by the real producers, never typed: page names and screenshots from the +# captured lab app (fixtures/look-shots, mxcli v0.28.0), the owed/seen ledgers by look-ledger.sh owe and +# the seen hook, ready.log / evidence / look-summary.tsv by bin/ready.sh, the "## Current stage" line in +# the exact form gate-check.sh derives, and the BUILD-LOG header and row in the shape exec.sh log_build() +# writes (bin/lib/build-log-read.sh quotes a real row: `| 2026-10-21T05:41:52+0000 | `x.mdl` | … |`). +# +# Usage: bash tests/wave2/test-ci-gates.sh [path-to-bin/ci-gates.sh] + +SUBJECT="${1:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/bin/ci-gates.sh}" +case "$SUBJECT" in /*) ;; *) SUBJECT="$PWD/$SUBJECT" ;; esac +TK="$(cd "$(dirname "$SUBJECT")/.." && pwd)" +FXS="$TK/tests/wave2/fixtures/look-shots" +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; } +WORK="$(mktemp -d "${TMPDIR:-/tmp}/citest.XXXXXX")" || exit 2 +trap 'rm -rf "$WORK"' EXIT +echo "== subject: $SUBJECT" +[ -f "$SUBJECT" ] && [ -d "$FXS" ] || { echo "FAIL — subject or fixture missing"; echo "PASS=0 FAIL=1"; exit 1; } +export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@t GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@t +export MXTK_NO_GUIDE=1 +G() { git -C "$1" "${@:2}"; } +hook() { printf '{"hook_event_name":"PostToolUse","tool_name":"Read","tool_input":{"file_path":"%s"},"tool_response":{}}' "$1"; } + +# mkproject — wired, three captured pages built through the real owe path +mkproject() { + mkdir -p "$1/bin" "$1/mdlsource" "$1/shots" "$1/docs/progress"; : > "$1/X.mpr" + cp "$TK/project-bin/look-ledger.sh" "$TK/project-bin/_common.sh" "$1/bin/" + printf '# PROJECT.md\n\n## Current stage\n\n**Stage P — Kickoff**, in progress.\n\nEntry mode: %s\n' "$2" > "$1/PROJECT.md" + printf '.claude/loop/\n' > "$1/.gitignore" + mkdir -p "$1/docs/progress/must-haves" # T1's mark, all rows TICKED: READY must not hinge on whether that skill has landed + printf 'Approval · 2 of 2 rows ticked or NOT DONE with a reason · pages 3 of 3\n1 TICKED — inbox lists open requests\n2 TICKED — decide records the decision\n' > "$1/docs/progress/must-haves/Approval.md" + printf 'CREATE PAGE "Approval"."Dashboard"\n(\n Title: %s\n) { }\ncreate or replace page Approval.Approval_Inbox (Title: %s) { }\nCREATE OR MODIFY PAGE Approval.Approval_Decide {\n}\n' "'Dashboard'" "'Approval inbox'" > "$1/mdlsource/20-approval-pages.mdl" + (cd "$1" && bash bin/look-ledger.sh owe "$1/mdlsource/20-approval-pages.mdl" >/dev/null) + G "$1" init -q; G "$1" add -A; G "$1" commit -qm scaffold +} +see_all() { + local s + for s in dashboard-1280:J-DASH-01-step1.png approval_inbox-1280:J-APP-01-step1.png approval_decide-1280:J-APP-01-step2.png; do + cp "$FXS/tests/e2e/artifacts/${s#*:}" "$1/shots/${s%%:*}.png"; hook "$1/shots/${s%%:*}.png" | (cd "$1" && bash bin/look-ledger.sh seen) + done +} +commit_all() { G "$1" add -A; G "$1" commit -qm "$2"; } +ci() { rm -rf "$WORK/ci"; git clone -q "$1" "$WORK/ci"; out="$(bash "$SUBJECT" "$WORK/ci" 2>&1)"; rc=$?; } + +# ── C0: clean — every page seen, READY by ready.sh, greenfield ──────────────────────────── +P="$WORK/c0"; mkproject "$P" greenfield; see_all "$P" +# READY needs P-4 passed or waived: greenfield waives 1-4 but owes 0, so the fixture joins at Build +# the way test-ready.sh does (--waive P, --adopt 5) +bash "$TK/bin/gate-check.sh" --no-html --waive P --reason "fixture: kickoff held elsewhere" "$P" >/dev/null 2>&1 +bash "$TK/bin/gate-check.sh" --no-html --adopt 5 --reason "fixture: joins at build" "$P" >/dev/null 2>&1 +commit_all "$P" "joins at build" +bash "$TK/bin/ready.sh" "$P" >/dev/null 2>&1 || bad "C0: ready.sh did not pass on a clean project" +ci "$P" +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q '^PASS ready' && printf '%s\n' "$out" | grep -q '^PASS look look: 3 of 3 pages seen' \ + && ok "C0: clean tree, READY written by ready.sh → CI passes" || bad "C0: rc=$rc $out" +[ ! -e "$WORK/ci/.claude/loop" ] && ok "C0: CI saw no ledger, only the committed summary" || bad "C0: the clone carries the ignored ledger" + +# ── C1 (b): READY written by hand ────────────────────────────────────────────────────────── +printf '# Progress\n\n| Module | State |\n|---|---|\n| Approval | READY |\n' > "$P/docs/progress/STATUS.md"; commit_all "$P" "hand status" +ci "$P" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL ready docs/progress/STATUS.md:5:' \ + && ok "C1: a hand-written READY table cell fails CI, file:line named" || bad "C1: rc=$rc $out" +G "$P" rm -q docs/progress/STATUS.md; G "$P" commit -qm "drop hand status" +# the field shape, verbatim but for the target and date (a field status file carried it five times, +# and the first version of the check passed it): a heading, READY followed by words +printf '# READY FOR BENCH (fix round 6, 2026-11-16 15:34 UTC)\n\nAll pages built.\n\n## READY FOR REVIEW\n' > "$P/docs/progress/status.md"; commit_all "$P" "heading claims" +ci "$P" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL ready docs/progress/status.md:1: # READY FOR BENCH' \ + && printf '%s\n' "$out" | grep -q '^FAIL ready docs/progress/status.md:5: ## READY FOR REVIEW' \ + && ok "C1: heading claims (# READY FOR BENCH ...) fail CI — the field shape" || bad "C1c: rc=$rc $out" +G "$P" rm -q docs/progress/status.md; G "$P" commit -qm "drop heading claims" +# same claim in a lowercase status.md at the ROOT (where the field copy kept it; the first scan matched STATUS*.md only) +printf '# READY FOR BENCH (fix round 6, 2026-11-16 15:34 UTC)\n' > "$P/status.md"; commit_all "$P" "root status" +ci "$P" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL ready status.md:1: # READY FOR BENCH' \ + && ok "C1: a lowercase status.md at the root is scanned too" || bad "C1d: rc=$rc $out" +G "$P" rm -q status.md; G "$P" commit -qm "drop root status" +# a hand-appended ready.log row with no evidence behind it +printf '2026-11-17T09:00:00Z\tREADY\tmodule Billing\tdocs/progress/ready-2026-11-17-Billing.md\t3 of 3\n' >> "$P/docs/progress/ready.log"; commit_all "$P" "forged log" +ci "$P" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'FAIL ready docs/progress/ready.log: READY module Billing' \ + && ok "C1: a READY row appended to ready.log by hand fails CI" || bad "C1b: rc=$rc $out" +G "$P" checkout -q HEAD~1 -- docs/progress/ready.log; G "$P" commit -qm "undo forged log" + +# ── C2 (c): READY, then a page built and never looked at ─────────────────────────────────── +sleep 1; printf 'CREATE OR MODIFY PAGE Approval.Approval_Archive {\n}\n' > "$P/mdlsource/21-archive.mdl" +(cd "$P" && bash bin/look-ledger.sh owe "$P/mdlsource/21-archive.mdl" >/dev/null && bash bin/look-ledger.sh summary --write >/dev/null) +commit_all "$P" "archive page" +ci "$P" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL look look: 3 of 4 pages seen — READY (app) while the committed look summary shows 1 built page(s) never looked at: Approval.Approval_Archive' \ + && ok "C2: READY standing over a committed unseen page fails CI" || bad "C2: rc=$rc $out" +# the same tree without a done-claim is a warning mid-build, a FAIL when strict +P2="$WORK/c2b"; mkproject "$P2" greenfield; (cd "$P2" && bash bin/look-ledger.sh summary --write >/dev/null); commit_all "$P2" summary +ci "$P2" +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q '^WARN look look: 0 of 3 pages seen — 3 built page(s) not looked at yet' && ok "C2: no claim → WARN, exit 0 (mid-build)" || bad "C2b: rc=$rc $out" +rm -rf "$WORK/ci"; git clone -q "$P2" "$WORK/ci"; out="$(MXTK_CI_STRICT_LOOK=1 bash "$SUBJECT" "$WORK/ci" 2>&1)"; rc=$? +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL look look: 0 of 3 pages seen — 3 built page(s) never looked at (strict)' && ok "C2: MXTK_CI_STRICT_LOOK=1 → FAIL" || bad "C2c: rc=$rc $out" + +# ── C3 (a): only stages the register records as PASSED fail; the stage in progress warns ───── +# Stage 5 in progress (gates passed: P-4), its look FAILs from the committed summary → a warning. +sed -i.bak 's/^\*\*Stage P — Kickoff\*\*, in progress\./**Stage 5 — Build**, in progress (manual gate: its verdict is a Decisions row, not a file) — gates passed: P, 0, 1, 2, 3, 4 (derived by gate-check on 2026-11-17; re-run it rather than editing this line)./' "$P2/PROJECT.md"; rm -f "$P2/PROJECT.md.bak" +commit_all "$P2" "claims stage 5" +ci "$P2" +printf '%s\n' "$out" | grep -q '^WARN stage Stage 5 is in progress and its gate FAILs .*3 of 3 built page(s) never looked at' \ + && ! printf '%s\n' "$out" | grep -q '^FAIL stage' && ok "C3: the stage in progress FAILing its gate is a warning, not a CI failure" || bad "C3: rc=$rc $out" +# The register says Stage 5 PASSED (Stage 6 in progress) while its gate FAILs → CI fails. +sed -i.bak 's/^\*\*Stage 5 — Build\*\*, in progress (manual gate: its verdict is a Decisions row, not a file) — gates passed: P, 0, 1, 2, 3, 4 /**Stage 6 — Test**, in progress — gates passed: P, 0, 1, 2, 3, 4, 5 /' "$P2/PROJECT.md"; rm -f "$P2/PROJECT.md.bak" +commit_all "$P2" "claims stage 5 passed" +ci "$P2" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL stage Stage 5 is recorded as passed in the register and its gate now FAILs: .*3 of 3 built page(s) never looked at' \ + && ok "C3: a stage recorded as passed whose gate FAILs fails CI (from the committed summary)" || bad "C3b: rc=$rc $out" + +# ── C4 (d): model work before Stage 4, requirements-driven ───────────────────────────────── +P3="$WORK/c4"; mkproject "$P3" requirements-driven +cat > "$P3/docs/BUILD-LOG.md" <<'LOG' +# Build log — auto-appended by bin/exec.sh + +One line per exec against the model. Written by the script, not by hand, so it is +true even when someone forgets. Read this before assuming the model builds. + +| when (ISO-8601 local) | script | gate | result | detail | +|---|---|---|---|---| +| 2026-10-21T05:41:52+0000 | `20-approval-pages.mdl` | pass | ok | 3 pages | +LOG +commit_all "$P3" "built early" +ci "$P3" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL order model work (1 exec row(s) in docs/BUILD-LOG.md, 1 build script(s) in mdlsource/ dirs) while Stage 4 is' \ + && ok "C4: requirements-driven model work before Stage 4 fails CI" || bad "C4: rc=$rc $out" + +# ── C7: enforcement off (what sync installs into an existing project): (d) warns, (b) still fails ─ +rm -rf "$WORK/ci"; git clone -q "$P3" "$WORK/ci"; out="$(MXTK_CI_ENFORCE=0 bash "$SUBJECT" "$WORK/ci" 2>&1)"; rc=$? +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q '^WARN order model work .*(advisory: set MXTK_CI_ENFORCE: "1"' \ + && ok "C7: MXTK_CI_ENFORCE=0 → the order check warns, exit 0" || bad "C7: rc=$rc $out" +printf '| Approval | READY |\n' > "$WORK/ci/STATUS.md"; out="$(MXTK_CI_ENFORCE=0 bash "$SUBJECT" "$WORK/ci" 2>&1)"; rc=$? +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL ready STATUS.md:1:' \ + && ok "C7: with enforcement off, a hand-written READY still fails CI" || bad "C7b: rc=$rc $out" + +bash "$TK/bin/gate-check.sh" --no-html --adopt 5 --reason "joined at build; stages before it were done elsewhere" "$P3" >/dev/null 2>&1 +commit_all "$P3" adopt +ci "$P3" +printf '%s\n' "$out" | grep -q '^PASS order' && ok "C4: an --adopt 5 point clears it" || bad "C4b: $out" + +# ── C5 (d): multi-app — build scripts per app, apps//mdlsource/ ─────────────────────── +P6="$WORK/c5"; mkdir -p "$P6/docs" +for a in Sales Stock; do mkdir -p "$P6/apps/$a/mdlsource"; : > "$P6/apps/$a/$a.mpr" + printf 'CREATE PAGE %s.Overview () { }\n' "$a" > "$P6/apps/$a/mdlsource/10-pages.mdl"; done +printf '# PROJECT.md\n\n## Current stage\n\n**Stage P — Kickoff**, in progress.\n\nEntry mode: requirements-driven\n' > "$P6/PROJECT.md" +G "$P6" init -q; commit_all "$P6" multi +ci "$P6" +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q '^FAIL order model work (0 exec row(s) in docs/BUILD-LOG.md, 2 build script(s) in mdlsource/ dirs)' \ + && ok "C5: per-app scripts (apps//mdlsource/) count as model work" || bad "C5: rc=$rc $out" + +# ── C6: the toolkit's own scaffold trips nothing — init, commit, CI ──────────────────────── +# (review of #788: the RESUME.md starter said "**READY is not written in this file.**", a claim by +# the check's own rule, so every new project was red on (b) and Stage 6 could never pass) +P7="$WORK/c6"; mkdir -p "$P7"; G "$P7" init -q +MXTK_NO_GUIDE=1 bash "$TK/bin/init-project.sh" "$P7" >/dev/null 2>&1 < /dev/null +grep -q 'MXTK_CI_ENFORCE: "1"' "$P7/.github/workflows/toolkit-gates.yml" 2>/dev/null && ok "C6: init installs the workflow, enforced" || bad "C6: no enforced workflow after init" +commit_all "$P7" "init" +ci "$P7" +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q '^PASS ready' && ! printf '%s\n' "$out" | grep -q '^FAIL' \ + && ok "C6: a fresh project's first push passes CI (ready PASS; Stage P in progress only warns)" || bad "C6: rc=$rc $out" + +# ── I: installer — create if absent, never overwrite ─────────────────────────────────────── +. "$TK/bin/lib/ci-template.sh" +P4="$WORK/inst"; mkdir -p "$P4/.github/workflows"; printf 'name: build\n' > "$P4/.github/workflows/build.yml" +m="$(mxtk_ci_install "$TK" "$P4" 0)" +cmp -s "$TK/templates/ci/toolkit-gates.yml" "$P4/.github/workflows/toolkit-gates.yml" && printf '%s' "$m" | grep -q 'existing workflows left untouched: build.yml' \ + && [ "$(cat "$P4/.github/workflows/build.yml")" = "name: build" ] && ok "I1: installed beside the repo's own CI, which is untouched" || bad "I1: $m" +printf '# adapted\n' >> "$P4/.github/workflows/toolkit-gates.yml" +m="$(mxtk_ci_install "$TK" "$P4" 0)"; tail -1 "$P4/.github/workflows/toolkit-gates.yml" | grep -q '^# adapted' && printf '%s' "$m" | grep -q '^Kept:' \ + && ok "I2: an adapted copy is kept and reported, never overwritten" || bad "I2: $m" +P6s="$WORK/inst3"; mkdir -p "$P6s"; m="$(mxtk_ci_install "$TK" "$P6s" 0 sync)" +grep -q 'MXTK_CI_ENFORCE: "0"' "$P6s/.github/workflows/toolkit-gates.yml" && printf '%s' "$m" | grep -q 'Installed advisory' \ + && [ -z "$(mxtk_ci_install "$TK" "$P6s" 0 sync)" ] && ok "I4: sync installs advisory (ENFORCE 0) with one notice; a re-run is silent" || bad "I4: $m" +P5="$WORK/inst2"; mkdir -p "$P5"; m="$(mxtk_ci_install "$TK" "$P5" 1)" +[ ! -e "$P5/.github" ] && printf '%s' "$m" | grep -q '^Would create' && ok "I3: dry run writes nothing" || bad "I3: $m" + +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ] diff --git a/tests/wave2/test-ready.sh b/tests/wave2/test-ready.sh new file mode 100755 index 00000000..6ed5c891 --- /dev/null +++ b/tests/wave2/test-ready.sh @@ -0,0 +1,186 @@ +#!/usr/bin/env bash +# test-ready.sh — bin/ready.sh: READY only when every check passes, K of N on every row, the +# evidence written and committed; plus the tracked look summary gate-check.sh reads from a copy. +# +# Golden input is captured, not imagined: the page names are the captured lab app's +# (fixtures/look-shots/show-pages-Approval.out, a finished Purchasing/Approval app, mxcli v0.28.0), +# the screenshots are that run's real PNGs, the owed ledger is written by the real +# `look-ledger.sh owe` from a build script in the captured statement shapes, and the seen ledger +# by the real `seen` hook from Claude Code's PostToolUse JSON. Nothing in .claude/loop/ is typed. +# +# Usage: bash tests/wave2/test-ready.sh [path-to-bin/ready.sh] + +SUBJECT="${1:-$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/bin/ready.sh}" +case "$SUBJECT" in /*) ;; *) SUBJECT="$PWD/$SUBJECT" ;; esac +TK="$(cd "$(dirname "$SUBJECT")/.." && pwd)" +FXS="$TK/tests/wave2/fixtures/look-shots" + +PASS=0; FAIL=0 +ok() { PASS=$((PASS+1)); echo " ok — $1"; } +bad() { FAIL=$((FAIL+1)); echo " FAIL — $1"; } +WORK="$(mktemp -d "${TMPDIR:-/tmp}/readytest.XXXXXX")" || exit 2 +trap 'rm -rf "$WORK"' EXIT +echo "== subject: $SUBJECT" +[ -f "$SUBJECT" ] && [ -d "$FXS" ] || { echo "FAIL — subject or fixture missing"; echo "PASS=0 FAIL=1"; exit 1; } + +# ready.sh commits with the caller's git identity; a CI runner has none +export GIT_AUTHOR_NAME=t GIT_AUTHOR_EMAIL=t@t GIT_COMMITTER_NAME=t GIT_COMMITTER_EMAIL=t@t +hook() { printf '{"hook_event_name":"PostToolUse","tool_name":"Read","tool_input":{"file_path":"%s"},"tool_response":{}}' "$1"; } +G() { git -C "$1" -c user.email=t@t -c user.name=t "${@:2}"; } + +# mkproject [mode] — a wired project: bin/ installed, three captured pages built +mkproject() { + mkdir -p "$1/bin" "$1/mdlsource" "$1/shots" "$1/docs/progress" + if [ "$2" = app ]; then mkdir -p "$1/app"; : > "$1/app/X.mpr"; else : > "$1/X.mpr"; fi + cp "$TK/project-bin/look-ledger.sh" "$TK/project-bin/_common.sh" "$1/bin/" + printf '# PROJECT.md\n\n## Current stage\n\n**Stage P — Kickoff**, in progress.\n\n%s\n' "${3:+Interview mode: $3}" > "$1/PROJECT.md" + printf '.claude/loop/\n' > "$1/.gitignore" + mkdir -p "$1/docs/progress/must-haves" # T1's mark, all rows TICKED: READY must not hinge on whether that skill has landed + printf 'Approval · 2 of 2 rows ticked or NOT DONE with a reason · pages 3 of 3\n1 TICKED — inbox lists open requests\n2 TICKED — decide records the decision\n' > "$1/docs/progress/must-haves/Approval.md" + cat > "$1/mdlsource/20-approval-pages.mdl" <<'MDL' +CREATE PAGE "Approval"."Dashboard" +( + Title: 'Dashboard' +) { } +create or replace page Approval.Approval_Inbox (Title: 'Approval inbox') { } +CREATE OR MODIFY PAGE Approval.Approval_Decide { +} +MDL + (cd "$1" && bash bin/look-ledger.sh owe "$1/mdlsource/20-approval-pages.mdl" >/dev/null) + # The position READY needs: stages P-4 PASS or WAIVED. The fixture joins at Build the way a real + # project records it, through gate-check's own --waive / --adopt (NOADOPT=1 skips this). + if [ -z "${NOADOPT:-}" ]; then + bash "$TK/bin/gate-check.sh" --no-html --waive P --reason "fixture: kickoff held elsewhere" "$1" >/dev/null 2>&1 + bash "$TK/bin/gate-check.sh" --no-html --adopt 5 --reason "fixture: joins at build" "$1" >/dev/null 2>&1 + fi + G "$1" init -q; G "$1" add -A; G "$1" commit -qm scaffold +} +# see — a screenshot named for its page, opened by the agent +see() { cp "$FXS/tests/e2e/artifacts/$3" "$1/shots/$2.png"; hook "$1/shots/$2.png" | (cd "$1" && bash bin/look-ledger.sh seen); } +see_all() { see "$1" dashboard-1280 J-DASH-01-step1.png; see "$1" approval_inbox-1280 J-APP-01-step1.png; see "$1" approval_decide-1280 J-APP-01-step2.png; } +# review +review() { + mkdir -p "$1/docs/progress/look-reviews" + cat > "$1/docs/progress/look-reviews/look-review-Approval.md" <&1)"; rc=$?; } + +# ── R1: owed but unseen pages → NOT READY, K of N, evidence committed ────────────────────── +P="$WORK/r1"; mkproject "$P" root +see "$P" dashboard-1280 J-DASH-01-step1.png +run "$P" +[ "$rc" = 1 ] && ok "R1: 1 of 3 seen → exit 1" || bad "R1: rc=$rc" +printf '%s\n' "$out" | grep -q 'FAIL look 1 of 3 page(s) looked at' && ok "R1: look row says 1 of 3" || bad "R1: $out" +printf '%s\n' "$out" | grep -q '^NOT-READY — 2 of 4 checks pass' && ok "R1: verdict NOT-READY with its denominator" || bad "R1: verdict: $out" +ev="$(ls "$P"/docs/progress/ready-*-app.md 2>/dev/null | head -1)" +[ -n "$ev" ] && grep -q '^- Approval.Approval_Inbox' "$ev" && grep -q 'Approval.Dashboard: dashboard-1280.png' "$ev" \ + && ok "R1: evidence names the unseen pages and the screenshot relied on" || bad "R1: evidence $(cat "$ev" 2>/dev/null)" +G "$P" ls-files --error-unmatch docs/progress/ready.log docs/progress/look-summary.tsv docs/progress/review-packets/app.html "${ev#"$P"/}" >/dev/null 2>&1 \ + && ok "R1: evidence, ready.log and look-summary.tsv are committed" || bad "R1: not committed: $(G "$P" status --short)" +grep -q '^as-of.*owed 3 seen 1 unseen 2' "$P/docs/progress/look-summary.tsv" && ok "R1: look-summary.tsv says owed 3 seen 1" || bad "R1: summary $(cat "$P/docs/progress/look-summary.tsv")" + +# ── R2: a copy of the repo (no ignored ledger) — gate-check reads the committed summary ──── +C="$WORK/r1-copy"; git clone -q "$P" "$C" 2>/dev/null +[ ! -e "$C/.claude/loop" ] || bad "R2: the clone has the ignored ledger" +g="$(bash "$TK/bin/gate-check.sh" --no-html "$C" 5 2>&1 | grep -m1 '^Stage 5 (')" +printf '%s' "$g" | grep -q 'FAIL.*2 of 3 built page(s) never looked at.*committed docs/progress/look-summary.tsv' \ + && ok "R2: a copy reports Stage 5 FAIL 2 of 3 from git, not MANUAL" || bad "R2: $g" +run "$C" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'look 1 of 3' && ok "R2: ready.sh on the copy is NOT READY, 1 of 3" || bad "R2: rc=$rc $out" + +# ── R3: all seen, steering → READY, the READY line written by the script ─────────────────── +see_all "$P" +run "$P" +[ "$rc" = 0 ] && ok "R3: all 3 seen → READY exit 0" || bad "R3: rc=$rc $out" +tail -1 "$P/docs/progress/ready.log" | grep -q ' READY app ' && ok "R3: ready.log carries the READY line" || bad "R3: log $(tail -1 "$P/docs/progress/ready.log")" +G "$P" log -1 --format=%s | grep -q '^ready: app READY (4 of 4 checks)' && ok "R3: committed as 'ready: app READY'" || bad "R3: $(G "$P" log -1 --format=%s)" +grep -q '^as-of.*owed 3 seen 3 unseen 0' "$P/docs/progress/look-summary.tsv" && ok "R3: summary refreshed to 3 of 3" || bad "R3: summary" + +# ── R4: unattended (auto) needs an independent review ────────────────────────────────────── +P="$WORK/r4"; mkproject "$P" root auto; see_all "$P" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'FAIL review 0 of 3 page(s) scored 4+' && ok "R4: auto, no reviewer file → NOT READY, review 0 of 3" || bad "R4: rc=$rc $out" +review "$P" other mdl-agent 3 +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'review 2 of 3 .*below 4: Approval.Approval_Decide=3' && ok "R4: a page scored 3 blocks, named" || bad "R4: $out" +review "$P" same mdl-agent 5 +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'ignored: needs distinct Reviewer: and Builder:' && ok "R4: a self-review (Reviewer = Builder) is not read" || bad "R4: $out" +review "$P" other mdl-agent 4 +run "$P" --no-commit +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q 'PASS review 3 of 3' && ok "R4: all seen + independent review 4+ → READY" || bad "R4: rc=$rc $out" +# a page rebuilt after the review is owed a new score (and a new look) +sleep 1; (cd "$P" && bash bin/look-ledger.sh owe "$P/mdlsource/20-approval-pages.mdl" >/dev/null); see_all "$P" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'review 0 of 3 .*unscored since last build' && ok "R4: a rebuild after the review voids its scores" || bad "R4: $out" + +# ── R5: --module scope; another module's unseen page does not block this one ─────────────── +P="$WORK/r5"; mkproject "$P" app; see_all "$P" +printf 'CREATE OR MODIFY PAGE Billing.Invoice_View {\n}\n' > "$P/mdlsource/30-billing.mdl" +(cd "$P" && bash bin/look-ledger.sh owe "$P/mdlsource/30-billing.mdl" >/dev/null) +run "$P" --module Approval --no-commit +[ "$rc" = 0 ] && printf '%s\n' "$out" | grep -q 'look 3 of 3' && ok "R5 (two-tree): --module Approval READY, 3 of 3" || bad "R5: rc=$rc $out" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'look 3 of 4' && ok "R5 (two-tree): the whole app is NOT READY, 3 of 4" || bad "R5: rc=$rc $out" +run "$P" --module Nope --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'look NOT MEASURED' && ok "R5: a module with no built page is NOT MEASURED, never green" || bad "R5: $out" + +# ── R6: the must-have mark (skills/must-haves.md § The mark), on its tracked path ────────── +mkdir -p "$P/docs/progress/must-haves" +printf 'Approval · 3 of 3 rows ticked or NOT DONE with a reason · pages 3 of 3\n1 TICKED — inbox lists open requests\n2 TICKED — decide records the decision\n3 NOT DONE — notification: no mail server in the lab\n' > "$P/docs/progress/must-haves/Approval.md" +run "$P" --module Approval --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'must-have 2 of 3 rows TICKED' && ok "R6: a NOT DONE row blocks READY, 2 of 3 (denominator from line 1)" || bad "R6: $out" +printf 'Approval · rows not counted yet\n1 TICKED — x\n' > "$P/docs/progress/must-haves/Approval.md" +run "$P" --module Approval --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'must-have FAULT — line 1 has no "N of M"' && ok "R6: a mark whose line 1 states no N of M is a FAULT, not a pass" || bad "R6: $out" +rm -f "$P/docs/progress/must-haves/Approval.md" + +# ── R6b: the review packet — paths, never images ─────────────────────────────────────────── +pk="$P/docs/progress/review-packets/Approval.html" +[ -f "$pk" ] && grep -q 'src="../../../shots/dashboard-1280.png"' "$pk" && grep -q '3 of 3 page(s) seen since their last build' "$pk" \ + && ! ls "$P"/docs/progress/review-packets/*.png >/dev/null 2>&1 \ + && ok "R6b: the packet lists each page's screenshot path and K of N, and holds no image" || bad "R6b: packet $(head -c 400 "$pk" 2>/dev/null)" + +# ── R7: no ledger at all → NOT MEASURED ──────────────────────────────────────────────────── +P="$WORK/r7"; mkdir -p "$P"; printf '# PROJECT.md\n' > "$P/PROJECT.md" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'look NOT MEASURED' && ok "R7: no ledger → NOT READY, look NOT MEASURED" || bad "R7: rc=$rc $out" + +# ── R8: every page seen, but no stage before Build ever passed → NOT READY (field run, 2026-11) ─ +P="$WORK/r8"; NOADOPT=1 mkproject "$P" root; see_all "$P" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'FAIL position .* of 6 stages before Build PASS or WAIVED — not: P ' \ + && printf '%s\n' "$out" | grep -q 'PASS look 3 of 3' \ + && ok "R8: Stage P not passed, all 3 pages seen → NOT READY on position" || bad "R8: rc=$rc $out" + +# ── R10: an annotated `Interview mode: auto (...)` line still owes the independent review ────── +# Field register line, verbatim but generic. interview-mode.sh falls back to steering on it; the +# review must not be waived by that fallback. +P="$WORK/r10"; mkproject "$P" root; see_all "$P" +printf -- '- Interview mode: auto (gate questions go to questions.md with an ASSUMED default)\n' >> "$P/PROJECT.md" +run "$P" --no-commit +[ "$rc" = 1 ] && printf '%s\n' "$out" | grep -q 'note review owed: PROJECT.md.s Interview mode line starts with "auto"' \ + && printf '%s\n' "$out" | grep -q 'FAIL review 0 of 3' && ok "R10: the strict reading owes the review; NOT READY without one" || bad "R10: rc=$rc $out" + +# ── R9: bounded time — every step has a watchdog; a stalled step is NOT MEASURED, never a hang ─ +# (field run, 2026-11: ready.sh on a six-app copy ran over 300 s with no output and was killed) +P="$WORK/r8" +start=$(date +%s); out="$(MXTK_READY_STEP_TIMEOUT=0 bash "$SUBJECT" "$P" --no-commit 2>&1)"; rc=$?; took=$(( $(date +%s) - start )) +[ "$rc" = 1 ] && [ "$took" -lt 60 ] && printf '%s\n' "$out" | grep -q 'gate5: gate-check stage 5: did not finish in 0s, killed' \ + && printf '%s\n' "$out" | grep -q 'FAIL gate5 NOT MEASURED' && printf '%s\n' "$out" | grep -q '^NOT-READY' \ + && ok "R9: every step killed at its timeout → NOT READY in ${took}s, each stalled step named" || bad "R9: rc=$rc took=${took}s $out" + +echo "PASS=$PASS FAIL=$FAIL" +[ "$FAIL" -eq 0 ]