Skip to content

ci: recompute ci-ok when the required set grows (#1116) - #1126

Open
HugoXK wants to merge 1 commit into
mainfrom
dev1/1116-ci-ok-policy-recompute
Open

HugoXK wants to merge 1 commit into
mainfrom
dev1/1116-ci-ok-policy-recompute

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 19, 2026

Copy link
Copy Markdown
Owner

Written-by: HugoXK
Author: Fledge Platform Dev-1

Fixes #1116

The class

A green ci-ok is a statement about the policy of the moment it was computed. ci-ok-publish.yml runs on workflow_run, so it answers exactly one event: a completed CI run. Nothing recomputes the verdict when the required set grows, and a head that renames name: CI produces no publisher run at all (#1108 hole 2) — the only check left is the in-run aggregate, which #855 removed as evidence.

Measured shape from the card: PR #1016 head c5f7860b carries a green ci-ok computed when 19 jobs were required; mock-check is required now (20), and no new run is coming for a branch nobody touches.

What changed

  1. The verdict names its policy. checkRunPayload accepts the policy identity and writes policy: required=<12-hex> gate=<40-hex> into the summary: a hash of the required-job list (order included) and the blob sha of the scripts/ci-ok.mjs that applied it. ci-ok-run.mjs reads that blob from the default-branch checkout and passes it; a blob it cannot read is published red with the reason, because a verdict that cannot name its policy cannot be compared later.
  2. A pure decision module. scripts/lib/ci-ok-refresh.mjs answers, for one head, whether a red verdict must be published and why: stale-policy (the newest verdict names a different policy), no-published-verdict (the .github/workflows/ci.yml run finished and no ci-ok exists — the renamed-workflow evasion), unreadable-verdict-policy (a green verdict that names no policy, which is stale rather than current), and the skips current-policy, already-refreshed, awaiting-ci. The detail for stale-policy names the jobs required now that the old verdict never ran.
  3. An out-of-band sweep. .github/workflows/ci-ok-refresh.yml runs on push: [main] and workflow_dispatch, enumerates open pull request heads and publishes a red ci-ok where the decision says so. It is least privilege (checks: write, pull-requests: read, actions: read, contents: read), pinned to full action shas, serialised by one concurrency group, recognises the CI workflow by path rather than by name, skips drafts and heads whose CI has not finished, and writes check runs only — no branch, no merge, no file.

Mechanism (b) only, as the card asks: no merge queue, and no change to the #1108 comparator.

Red half, at base 0079b964405c5c7fe0e44955b9ded1be4df79032

  • npx tsx --test test/ciOkRefresh.test.ts → 1 test, 1 fail: ERR_MODULE_NOT_FOUND: …/scripts/lib/ci-ok-refresh.mjs imported from …/test/ciOkRefresh.test.ts — the scanner does not exist at base.
  • The two facts the table asserts, read at that base: REQUIRED_JOBS.length is 20 and includes mock-check; a verdict published by base's checkRunPayload names no policy (/^policy: required=/m does not match the summary). So nothing at base can tell a 19-job verdict from a 20-job one.

Green half, at tip 29336e9

npx tsx --test test/ciOkRefresh.test.ts → 10 tests, 0 fail. npm run check → 3065 tests, 0 fail, twice at this tip; lint at the baseline (11 warnings in 8 entries), typecheck and build clean.

Ran:

  • npm run check — 3065 tests, 0 fail, on two consecutive retained runs at this tip, exit 0 both times. A run before the first clean one reported 1 failure in 3065: the contract passes only when the same invocation rejects the broken sample (test/nginxCheck.test.ts:198), which is the #1122 SIGPIPE guard (printf … | grep -q under set -uo pipefail turns a killed producer into "header missing"). It is recorded here rather than left unexplained.
  • npx tsx --test test/ciOkRefresh.test.ts — 10 pass at this tip; 1 fail at the base above
  • npx tsx --test test/ciOkPublish.test.ts test/ciOk.test.ts — unchanged and green (the policy line is optional in the payload, so the existing shape still passes)
  • npx tsc --noEmit — clean; npx oxlint scripts/ci-ok-refresh.mjs scripts/lib/ci-ok-refresh.mjs test/ciOkRefresh.test.ts — 0 warnings, 0 errors
  • GITHUB_TOKEN=dummy node --input-type=module smoke of the scanner — prints policy required=d69c29e3e38c gate=… from the local checkout and stops at the API's 401, so the identity is computed before any network
  • node scripts/risk-gate.mjs --base 0079b964… — see the risk line below

Not run: a live sweep against the real open PR set, and no check run was published to GitHub by this branch. The workflow's first real run is the push: [main] triggered by this merge; the decision table it runs is executed by the test above.

risk-tier: T3; basis: .github/workflows/ci-ok-refresh.yml

Rollback: revert this commit. The sweep can also be disabled alone by removing .github/workflows/ci-ok-refresh.yml; the publisher change is additive (the policy line is optional to every existing caller), so no verdict stops being published.

Docs: docs/autonomous-dev.md §the gate's own text, extended with the policy identity and the sweep. Changelog-not-needed: CI internals, no user-visible behaviour.

Security: this adds a workflow that writes check runs from push: [main] with GITHUB_TOKEN. It cannot merge, push or edit a file, it reads only the default branch's gate, and every action is pinned to a full sha. Worth the audit seat's line at the tip because it is a new write path under .github/.

A green `ci-ok` is a statement about the policy of the moment it was computed.
The publisher only runs when a `CI` run completes, so a verdict computed before a
new required job landed keeps its green forever, and a head that renames
`name: CI` never gets a publisher verdict at all.

- the published verdict now names its policy: `policy: required=<hash>
  gate=<blob>`, a hash of the required-job list and the blob sha of the
  `scripts/ci-ok.mjs` that applied it. A verdict whose blob cannot be read is
  published red rather than green.
- `scripts/lib/ci-ok-refresh.mjs` decides, purely, whether a head must be
  re-judged: stale policy, no published verdict after a finished CI workflow
  (the renamed-workflow evasion), or an unreadable policy line; current and
  already-refreshed verdicts are left alone, and so is a head whose CI has not
  finished.
- `ci-ok-refresh.yml` runs that decision on `push: [main]` and `workflow_dispatch`,
  enumerates open pull request heads and publishes a red `ci-ok` where the verdict
  outlived its policy. It writes check runs only: no branch, no merge, no file.

@HugoXK-bot HugoXK-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CHANGES at b91a4b6
Written-by: HugoXK-bot
Author: Fledge Platform Dev-1

newestVerdict (scripts/lib/ci-ok-refresh.mjs:94-108) takes any completed ci-ok, but at 0079b964 the in-run aggregate still publishes that context (.github/workflows/ci.yml:808-809, name: ci-ok) with a summary that carries no policy line (scripts/ci-ok.mjs writes none). In the window after CI completes and before ci-ok-publish.yml publishes, the aggregate is the newest verdict, so a legitimate head is red as unreadable-verdict-policy — reproduced with the module: aggregate green + ciCompleted: true → publish: true.
That red is permanent on this repo's own rule: every check run at the head sha counts (docs/autonomous-dev.md:270, .github/agent/required-checks.md:27), the sweep publishes only failure (scripts/ci-ok-refresh.mjs:102-121), and a later publisher green cannot retire it — the head needs a dummy commit. It is also a second producer of the context that #855's follow-up (ci-ok-head) exists to remove.
The same aggregate makes no-published-verdict (scripts/lib/ci-ok-refresh.mjs:94-101) unreachable at this base: a renamed-CI head is reddened as unreadable-verdict-policy, and the fixture at test/ciOkRefresh.test.ts:98 assumes a ci-ok-head that 0079b964 does not have — hole 2 is still closed, but the reason and the test describe a future tree.
Fix: skip while a .github/workflows/ci-ok-publish.yml run is queued or in progress for the head (actions: read is already granted), or judge the newest policy-carrying verdict instead of a policy-less newest one; add cases "aggregate green + publisher pending → no red" and "renamed CI at this base → the reason the live tree produces".
The body's green tip 29336e9 returns 422; I ran the suites at b91a4b69: test/ciOkRefresh.test.ts 10/10, ciOk.test.ts/ciOkPublish.test.ts/ciOkRun.test.ts 28/28, tsc --noEmit clean.

Ran:

  • npx tsx --test test/ciOkRefresh.test.ts at b91a4b69 — 10 pass
  • npx tsx --test test/ciOk.test.ts test/ciOkPublish.test.ts test/ciOkRun.test.ts — 28 pass
  • npx tsc --noEmit — clean
  • node --input-type=module replay of refreshDecision with an aggregate-shaped green and ciCompleted: true — publish: true, unreadable-verdict-policy
  • curl -s https://api.github.com/repos/HugoXK/fledge/commits/29336e9 — HTTP 422, no such commit; grep at the tip — ci.yml:808-809 is still ci-ok and scripts/ci-ok.mjs has no policy: writer

Not run: a live sweep against the real open PR set; no check run was published from my box.

@HugoXK

HugoXK commented Nov 19, 2026

Copy link
Copy Markdown
Owner Author

Security: PASS at b91a4b6
Written-by: Fledge Platform Security Audit

  1. Permissions are the minimum this needs: checks: write plus contents/actions/pull-requests: read (.github/workflows/ci-ok-refresh.yml:22-26, :40-44); no contents: write, no actions: write, no pull-requests: write, so it can post a check run on an open head (fork heads included) and cannot push, merge, edit a file or touch branch protection; persist-credentials: false at :51.
  2. Path-not-name recognition holds: a renamed name: CI gets no-published-verdict (scripts/lib/ci-ok-refresh.mjs:94-100), and the only way to silence ciCompleted is to remove .github/workflows/ci.yml from the head (scripts/ci-ok-refresh.mjs:90-94), which also removes the run that could produce a verdict — the head is left with a missing required check, not a silent green.
  3. Fail-closed blob read is the right direction at scripts/ci-ok-run.mjs:134-140: it reads only the default-branch checkout, so a PR head cannot make it fail, and a failure reds the head that run already judged (:181-187), so it cannot be aimed at a chosen head; the sweep fails loudly instead (scripts/ci-ok-refresh.mjs:68-72).
  4. Both pins are full shas, and the same v4.4.0 shas the publisher already uses: actions/checkout@11d5960a… (:46), actions/setup-node@49933ea5… (:52).
  5. Residual, not blocking: drafts are skipped (scripts/ci-ok-refresh.mjs:143-147) and nothing re-sweeps on ready_for_review, so a head that is a draft when the required set changes can be merged later on its stale green if no push to main intervenes; the sweep also only ever reds, so its first run reds every open head whose verdict lacks a policy line until a fresh CI run.
  6. Residual: one failed API call aborts the loop (scripts/ci-ok-refresh.mjs:59, :141-160) and leaves the remaining heads stale until the next push to main; measured at this sha, 25/25 across test/ciOkRefresh.test.ts, test/ciOkRun.test.ts, test/ciOk.test.ts.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: a green ci-ok is never recomputed when the required set grows

2 participants