Skip to content

new(ready, ci): READY only via bin/ready.sh, evidence in git, and a CI gate an agent cannot skip - #788

Merged
MendixMau merged 9 commits into
masterfrom
claude/ready-command
Nov 17, 2026
Merged

MendixMau merged 9 commits into
masterfrom
claude/ready-command

Conversation

@MendixMau

@MendixMau MendixMau commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

Direct lane

What changed and why: Two unattended field runs (2026-11) show the same failure. In one, a build agent wrote READY by hand with 0 of about 78 pages ever looked at. The look ledger would have FAILed that, but nobody ran the gate. Because .claude/loop/ is git-ignored, a copy of the repo showed nothing. In the other, a different kit marked itself ready 3 hours early, with no review pass. Everything that runs inside the agent's session can be skipped by the agent. This PR does three things:

  • Only one command can write READY.
  • The evidence behind READY goes into git.
  • A CI gate runs outside the session, on what was pushed.
Commit What
3e9302c ready.sh, the tracked look summary, Stage 5 reading it
c186bf7 CI gate, READY-claim check, review packet, must-have mark, multi-app (owner's mid-run design input)
e1a7ac6 READY needs stages P–4 passed; must-have waiver / ADVISORY; visible look count in CI
15618fd Fixture git identity t@t (the leak guard blocks email-shaped strings)
e10cf31 Fixes from the field run on a six-app copy: heading READY claims, a watchdog on every step, per-app scripts, and Stage 5's unbound rows (my regression from 3e9302c)
8e0a4e9 Fixes from the field re-run: lowercase root status.md; strict reading of Interview mode for the review
3d30264 Fixes from the independent review (MERGE-AFTER-FIX): the toolkit's own text tripped its READY check; (a) fails only on stages recorded as passed; MXTK_CI_ENFORCE (synced projects start advisory); test-ci-gates.sh C0/C4

bin/ready.sh <root> [--module M | --app A] [--no-commit]

Exit 0 means READY, 1 means NOT READY. Every row prints K of N. A row that measured nothing reports NOT MEASURED and fails.

Row Rule
position Stages P and 0–4 must each be PASS, WAIVED (an --adopt point) or not owed by the entry mode, from one read-only full gate-check run. A field run wrote READY with Stage P never passed. --adopt 5 waives 0–4 but not P.
gate5 PASS or WAIVED, or MANUAL with the look accounted for. --module scopes it through MXTK_LOOK_MODULE.
gate6 Owed only once the register's ## Current stage is at Stage 6 or later.
look Uses gate-check's own join: opened screenshots, PROOF-OF-LOOK citations and waivers all count.
must-have T1's mark, read from the tracked path docs/progress/must-haves/<Module>.md. K = the TICKED rows; N = line 1's first "N of M". A line 1 with no "N of M" is a FAULT. Waived obligation must-haves[/M] reads WAIVED. An Obligation must-haves ADVISORY verdict is shown but does not block. A missing mark blocks once skills/must-haves.md exists.
review Owed under unattended mode. It is owed when interview-mode.sh says auto, or when the raw Interview mode: line's first word is auto/unattended; the stricter reading wins, and the script prints one line saying so. It reads docs/progress/look-reviews/*.md and requires Reviewer: ≠ Builder:. Every page needs a score of 4 or higher, given after its last build.
app: In a multi-app project every app is named NOT MEASURED for now (follow-up 4); it never reports "nothing checked".

Bounded time:

  • Every external step names itself on stderr before it starts.
  • It runs with stdin closed.
  • It is killed after MXTK_READY_STEP_TIMEOUT (default 120 s), and its row then reads NOT MEASURED.

The field copy took over 300 s on c186bf7 and 35 s on e10cf31.

Files it writes (all tracked, no images):

  • docs/progress/ready-<date>-<scope>.md: the evidence.
  • docs/progress/ready.log: the READY line.
  • docs/progress/review-packets/<scope>.html: per module, each page's screenshot beside its target with the reviewer's score, under the K-of-N lines. Image paths only. It sits in a subfolder because report-disposition-check.sh reads docs/progress/*.html as reports.
  • docs/progress/look-summary.tsv.

It then commits them with an explicit pathspec.

A READY that ready.sh did not write is not READY (bin/lib/ready-claims.sh)

What counts as a claim: a line in a status file whose first word is READY (after heading, list, bold or quote markup, or a Status:/State:/Verdict: label), whatever follows it, or a table cell that is exactly READY. Field shape: # READY FOR BENCH (fix round 6, …), eight times in a root status.md.

Which files are status files:

  • PROJECT.md;
  • root resume, status* and progress* .md files, in any letter case;
  • every .md under docs/ except ready.sh's own output.

When a claim is backed: when it names a docs/progress/ready-*.md whose verdict line says READY and which ready.log lists as READY. Each scope's latest ready.log row is held to the same test.

Stage 6 FAILs on any unbacked claim. The toolkit's own text no longer starts a line with READY; fixture C6 runs init → commit → CI. The RESUME.md starter and the ready routing row (in every CLAUDE.local.md and all six agent stubs) state the rule.

The CI gate: bin/ci-gates.sh + templates/ci/toolkit-gates.yml

init-project.sh and sync-project.sh install .github/workflows/toolkit-gates.yml:

  • create-if-absent, never overwritten (an adapted copy is kept and reported);
  • installed beside any existing CI, which is left untouched.

It runs on every push and PR, using the toolkit at the commit PROJECT.md pins.

Enforcement (owner policy). init installs MXTK_CI_ENFORCE: "1". sync installs "0" with one notice, so in a project the toolkit reached mid-flight, (a) and (d) warn until the project switches them on. (b) fails the build either way.

Check Fails when
(a) stage a stage the register records as passed (gates passed: P, 0, …) now FAILs its gate. The stage in progress only warns, so a fresh project at Stage P passes its first push.
(b) ready any unbacked READY claim (always enforced)
(c) look unseen built pages while something claims done (a READY row covering them, or Stage 6 or later), or a READY with no committed summary. Otherwise it prints look: K of N pages seen as a warning. MXTK_CI_STRICT_LOOK=1 fails always.
(d) order model work (BUILD-LOG exec rows, or .mdl scripts in mdlsource/ and apps/<App>/mdlsource/) while Stage 4 is not PASS or WAIVED

It reads files only and needs no runtime; a run takes about 1.5 s. The template header lists what it cannot see.

Follow-ups (not in this PR)

  1. interview-mode.sh and annotated lines. It resolves - Interview mode: auto (gate questions go to questions.md …) to steering, because an unparseable line falls back to asking more. That is right for questioning, but every other reader of the mode inherits a relaxed gate. ready.sh works around it with the strict raw-line reading. The resolver should accept a trailing annotation, or every gate reader should take the stricter of the two readings.
  2. One built-page count (after fix(obligations): quality floor — per-module passes owed by what was built; one must-have list for every build type #787 merges). ready.sh's look row counts pages from the look ledger or summary, so a field copy with no ledger says "no page recorded as built" while T1 counts 78 pages from the scripts. After both merge, ready.sh should use T1's built-page count as the denominator, or report NOT MEASURED naming the script count.
  3. For T2 (exec.sh): call look-ledger.sh summary --write after owe, and add one line to the BUILD-LOG header saying READY is not recorded there.
  4. Per-app scope (after new(multi-app): apps/{App}/{App}.mpr as a supported layout — per-app resolver, gate-check, status, look-ledger, scaffold #789 merges). The app:<A> NOT MEASURED rows go away once ready.sh switches to the per-app look ledger.

Impact

Row Answer
Process Blocks fall on claims, never on writes. READY needs a passing ready.sh, including stages P–4 and, unattended, an independent reviewer file. An unbacked READY FAILs Stage 6 and CI. New projects are bound from their first push. Synced projects get the workflow advisory (a hand-written READY still fails), plus one notice saying how to enforce.
Dependencies Touches gate-check.sh (Stage 5 summary fallback and module scope; Stage 6 claim check), look-ledger.sh, close-task.sh, init-project.sh, sync-project.sh (4a1), resume-template.sh, skill-routing.tsv plus its re-render. Adds bin/ready.sh, bin/ci-gates.sh, bin/lib/ready-claims.sh, bin/lib/ci-template.sh, templates/ci/toolkit-gates.yml. Composes with T1 #787 (mark path agreed), T4 #789 (mxtk_apps, per-app ledger) and T2 (exec.sh follow-ups).
Context cost Baseline 64,984 → 65,100 words (+116, budget 80,000). Stage 5 lead 27,330 → 27,446; Stage 6 lead 27,026 → 27,142; other roles unchanged. All of it is the dispatching.md paragraph.
Behaviour NOT RUN: no A/B on the dispatching paragraph.
Runtime Time cost: per READY claim · ~1.7 s here, 35 s on the six-app field copy · per CI run ~1.5 s · can force rework: y · offset: replaces a human audit of "did anyone look".

Field evidence: a field run on a real copy of a six-app field repo (tracked files only), three passes.

Defects the first two passes found, all fixed here:

  • heading READY claims and a lowercase root status.md not scanned (both had passed by absence);
  • ready.sh running over 300 s;
  • per-app scripts uncounted;
  • the review waived by an annotated mode line.

Final pass on 8e0a4e9: passes.

  • CI ready FAILs all 8 hand-written # READY FOR BENCH … lines in the root status.md.

  • CI order counts 276 scripts and FAILs: model work before Stage 4.

  • ready.sh prints the review-owed note and reads NOT-READY, 0 of 10, with position listing P FAIL and 0–4 PENDING. The run ends in bounded time.

  • No client data in the diff

  • Size cap: over (about 24 files; eight are generated re-renders, two are fixtures), because the owner's follow-ups landed here. I can split the CI half into a stacked PR if preferred.

  • Test tier: I did not run the fixtures locally; the sandbox blocked fixture runs. I replayed the new and changed cases by hand: C3 in-progress WARN and recorded-pass FAIL; C6 fresh init → commit → CI exit 0 with (b) PASS; C7 enforcement off. CI runs both suites. The reviewer found test-ci-gates.sh 3 of 17 red on 8e0a4e9; C0 and C4 are fixed in 3d30264, and C2 was a knock-on of C0. Reviewer: re-run test-ready.sh, test-ci-gates.sh, test-look-ledger.sh, test-bug12-sync.sh and test-dryrun-writes-nothing.sh.

  • Golden input from real producers (captured lab-app pages and PNGs, real owe/seen/ready.sh, BUILD-LOG in exec.sh's shape, the field claim lines genericised, a real init-project.sh scaffold), both layouts plus multi-app. Bash 3.2 and BSD date fallbacks in place; not run on macOS.

  • Routing row added and re-rendered

  • CHANGELOG lines

Do not merge before an independent review.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y

bin/ready.sh runs gate-check Stage 5 (Stage 6 from the register's position),
the look ledger, the must-have list when present and, under Interview mode
auto, an independent reviewer's scored verdict file. K of N per check;
exits non-zero unless all pass; writes the READY line itself and commits
docs/progress/ready-<date>-<scope>.md with explicit paths.

look-ledger.sh summary --write puts the git-ignored ledger into a tracked
docs/progress/look-summary.tsv (ready.sh, close-task.sh); gate-check Stage 5
reads it when the ledger is absent, so a copy of the repo FAILs, not MANUAL.

Field evidence: two unattended field runs, 2026-11 — READY hand-written with
0 of ~78 pages looked at; another kit marked ready 3 hours early, no review.
Exercised against a project rebuilt from the captured lab-app pages and
screenshots (tests/wave2/fixtures/look-shots), both layouts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y
…rite

bin/ci-gates.sh runs in the project's CI (templates/ci/toolkit-gates.yml,
installed create-if-absent by init and sync) over the committed tree at the
pinned toolkit commit. It fails on the claimed stage's gate FAIL, on a READY
line bin/ready.sh did not write (bin/lib/ready-claims.sh, also a Stage 6
FAIL), on committed unseen pages under a done-claim, and on model work before
Stage 4. It takes seconds and needs no Mendix runtime.

ready.sh also writes a review packet (paths only, never images), reads T1's
must-have mark on the tracked path docs/progress/must-haves/<Module>.md, and
names every app of a multi-app project instead of checking nothing.

Field evidence: the same two unattended field runs (2026-11). Exercised on
fresh clones of projects built from the captured lab-app pages/screenshots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y
@MendixMau MendixMau changed the title new(ready): READY only via bin/ready.sh, evidence committed to git new(ready, ci): READY only via bin/ready.sh, evidence in git, and a CI gate an agent cannot skip Nov 17, 2026
…ble look count

ready.sh adds a position row: stages P and 0-4 must each be PASS, WAIVED
(an --adopt point) or not owed by the entry mode, from one read-only full
gate-check run. A field run wrote READY with Stage P never passed; a look
accounted for in Stage 5 is not enough on its own. ready.sh also honours
`Waived obligation must-haves[/<Module>]` and shows an ADVISORY obligation
verdict without blocking. ci-gates.sh prints "look: K of N pages seen" on
every look line, as a warning when it does not fail.

Fixtures join at Build through gate-check's own --waive P / --adopt 5; new
R8: Stage P not passed, every page seen -> NOT READY.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y
… guard blocks email-shaped strings)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y
…ps, per-app scripts

From the parent's field run on a six-app copy:
- ready-claims: READY as the first word counts whatever follows it, so
  "# READY FOR BENCH (fix round 6, ...)" (five times in a field status file)
  is a claim. Captured shape added to test-ci-gates C1.
- ready.sh: every external step names itself on stderr, runs with stdin
  closed under a watchdog (MXTK_READY_STEP_TIMEOUT, default 120 s) and
  reports NOT MEASURED when killed; the step subshell clears the EXIT trap
  so a kill cannot delete $TMP. Not reproduced here (6 apps, 430 BUILD-LOG
  rows: 1.5 s); the progress lines will name the step on the re-run. R9.
- ci-gates order: counts apps/<App>/mdlsource/ via mxtk_apps or the glob. C5.
  Look now WARNs when exec rows exist but no look summary is committed.
- gate-check Stage 5: rows="" — unbound under set -u on a project with no
  ledger and no summary, which emptied the verdict (regression from 3e9302c).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y
…ode for the review

Field re-run on e10cf31 (ready.sh finished in 35 s; hang gone):
- ready-claims scanned STATUS*.md only; the field claims sit in a lowercase
  root status.md. Root status*/progress*/resume .md now match in any case. C1d.
- `- Interview mode: auto (gate questions go to questions.md ...)` resolves to
  steering in interview-mode.sh (unparseable -> ask more), which waived the
  independent review. ready.sh now owes the review when the raw line's first
  word is auto/unattended, and says so on one line. interview-mode.sh is not
  changed here. R10.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y
… recorded passes, advisory sync

Independent review (MERGE-AFTER-FIX):
1. resume-template.sh and close-task.sh started a line with READY, which the
   claim check matches: every fresh project was red on (b) and Stage 6 could
   never pass (guard rule 7). Reworded ("Only bin/ready.sh writes READY").
   New fixture C6: init -> commit -> ci-gates, (b) PASS, exit 0.
2. test-ci-gates.sh: C0 joins at Build with --adopt 5 (greenfield owes
   Stage 0); C4 expects the "mdlsource/ dirs" wording.
3. Check (a) FAILs only on stages the register records as passed ("gates
   passed: ..."), one full read-only gate-check run; the stage in progress
   WARNs. C3 split into both cases.
4. MXTK_CI_ENFORCE: init installs "1", sync installs "0" with one notice; at
   "0" (a) and (d) warn, (b) still fails. Fixtures C7 and I4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y
CI on 8e0a4e9 reaped six orphaned `sleep` processes after the fixture run:
when a step finished first, killing the watchdog subshell left its bare
`sleep 120` running. The sleep now runs as the watchdog's child and a TERM
trap kills it. Timeouts unchanged (checked: a stalled step is still killed
at MXTK_READY_STEP_TIMEOUT, rc 124; no sleep survives a normal run).

That CI run otherwise: 227 of 228 fixtures pass, test-ready.sh 27/27; the one
failure (test-ci-gates C0/C2/C4) is the review finding fixed in 3d30264.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXpQSjAZQFu8tasj2BMz5Y
@MendixMau
MendixMau merged commit 90c3e66 into master Nov 17, 2026
2 checks passed
MendixMau pushed a commit that referenced this pull request Nov 17, 2026
…eenfield waiving stages 1-4

test-ci-gates.sh C0/C2b built projects as `Entry mode: greenfield` and relied
on its stage 1-4 waiver; they are requirements-driven now, C2b adopted at 5
(the reviewer's trial: 21/1 -> 22/0). Comments in bin/ci-gates.sh and the
resume template's Q1 text name the two doors instead of greenfield.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HXpQSifRwo7EWWDP1hPps8
MendixMau added a commit that referenced this pull request Nov 17, 2026
…n rule, design front end (#790)

With no declared entry mode or adoption, every exec.sh run, status.sh and the gate-check summary print one warn-only banner: the toolkit treats the project as Stage P, not started. In migration and requirements-driven projects, the first MDL write with no plan evidence is refused once per clone, before any lock, snapshot or write. It clears with an adoption at stage 5 or later, a reasoned stage-4 waiver (including the aggregate `Waived stages: … 4 — reason`), a Stage 5+ readout, or a build plan plus a brief. Never refused: a-la-carte (new), poc, existing-app, --patch and undeclared projects. Documentation adds the greenfield design front end (poc-build steps 1–4) and mapping a supplied prototype as the wireframe (design-artifacts Step 2a; the ui-preflight STOP accepts it). project-bin/_position.sh is the shared, process-free tokeniser.

Field evidence: two unattended requirements-driven field runs built about 78 pages with no plan while every guard stayed silent.

Review: MERGE — refuse-once plan rule and banner verified (46/46, bug06 64/0, 0 process starts added); composes with #787 and #788 (65,373 words); CI red once on unrelated flaky test-app-restart.sh (mktemp suffix vs "zqx"), green on re-run

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MendixMau added a commit that referenced this pull request Nov 17, 2026
…nts-driven) (#792)

Greenfield is retired as an entry mode. A project goes through one of two doors: POC, which reads a requirements doc and builds a demo, or requirements-driven, which is the full stage pipeline for a real app. An existing register that still says greenfield keeps working: the token is read as retired and held to the same unattended plan rule as requirements-driven, so it can no longer bypass the build-ready refusal. The fixtures that covered greenfield, including #788's test-ci-gates, follow the retirement.

Review: MERGE — retired greenfield held to the auto plan rule; fixtures (incl. #788's test-ci-gates) follow the retirement (head 8400aaf tree identical to reviewed 8dec9dc; CI green)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MendixMau added a commit that referenced this pull request Nov 17, 2026
…ion check (#793)

The composition check of #787/#788/#790/#791/#792 found three ways an unattended agent could still get past the plan gate. This closes them:
- Auto detection uses one first-word rule (mxtk_auto_word in project-bin/_position.sh) shared by exec.sh, gate-check.sh's --adopt check and ready.sh. `Interview mode: auto (…)` and `unattended — …` now count as auto, and the strictest of env, state file and register wins.
- ready.sh reports NOT READY while any `consentBy: auto-mode` adoption row is unconfirmed, so an agent cannot approve its own plan skip and reach READY.
- Under auto, an undeclared Entry mode is refused before build-ready. Attended behaviour is unchanged.
- The checkpoint skills no longer describe greenfield as a live mode.

Review: MERGE — no findings (the H/K/W/U order fix restores test-mode-contract to 28/0; the strictest-wins change is now in the CHANGELOG; CI green on d3c4ced)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MendixMau MendixMau mentioned this pull request Nov 17, 2026
MendixMau added a commit that referenced this pull request Nov 17, 2026
This is the output of bin/cut-release.sh. The 24 entries under `## Unreleased` become `## v2026.11.17`. Included: #787, #788, #790, #791, #792 and #793. #789 is held.

Review: MERGE — no findings (CHANGELOG heading only; the release section has no client names; CI green on 072e417)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants