Repository navigation
Leave CI result to a newer pr.yml run for this pull request, and read the live draft state (#6656) - #6681
Conversation
A push and then marking the PR ready within seconds starts two pr.yml runs on one head, and cancel-in-progress cancels the first. Its verdict job still ran "Set CI result" and posted failure on that head until the newer run overwrote it. The step now skips when the workflow run is cancelled, so the newer run owns the status.
There was a problem hiding this comment.
Verdict: LGTM
Claims
- VERIFIED — the verdict job's job-level
ifisalways() && (...), so it still runs when upstream jobs are cancelled/failed — matches.github/workflows/pr.yml:335(pre-existing, unchanged by this diff) - VERIFIED — "the step posts nothing" when the run is cancelled — matches the new
if: ${{ !cancelled() }}at.github/workflows/pr.yml:342 - VERIFIED — "a test leg that times out does not cancel the workflow, so the step still posts failure" — a timed-out
Run test legstep (.github/workflows/pr.yml:302-309,timeout-minutes: 25) makes thetestjob's own conclusionfailure, notcancelled; that is a different job's result flowing intoTEST_RESULT, not a cancellation signal on theverdictjob itself, socancelled()stays false and the step still runs - VERIFIED — "the dev is red step after it already requires
steps.verdict.outputs.state != '', so it skips when the verdict step is skipped" — matches.github/workflows/pr.yml:392 - VERIFIED — "the verdict step is the job's first step" — confirmed structurally, it's the first entry under the job's
steps:(.github/workflows/pr.yml:338-339) - UNVERIFIED — "a push to a draft PR... Marking the PR ready about seven seconds later starts a second run on the same head" — a live timing claim about two specific Actions runs; not checkable from a read-only tree, would need the Actions run history/timestamps for #6637
- UNVERIFIED — "Seen 2026-10-23 on #6637, run 40033237161" — a specific historical run id; would need to query that run via the Actions API/UI
Riskiest lines
.github/workflows/pr.yml:342— the new guard itself; if it over-fires it would silently swallow a genuine failure, if it under-fires the original bug (a cancelled run posting a bogusfailure) persists — UNCOVERED (no behaviour test can exercise GitHub Actions' own cancellation semantics from this repo; the PR body gives the one-sentence reason: "proven by CI runs on this PR," and the pattern is already established elsewhere in this repo —build.ymlandwpf-smoke.ymlboth useif: ${{ !cancelled() }}the same way).github/workflows/pr.yml:386— thegh api .../statusescall the new guard gates; a wrong guard surfaces here as a wrongly-posted or wrongly-withheld commit status — UNCOVERED, same reasoning as above.github/workflows/pr.yml:392— unchanged downstream step whosesteps.verdict.outputs.state != ''check is the safety net the PR body leans on if the new guard skips the step; re-read and confirmed it still holds (the output simply stays unset when the step is skipped) — not itself at risk from this diff, but it's the line that makes the change safe
Traps near this diff
- "dorny/paths-filter v4 evaluates each pattern as an INDEPENDENT predicate..." — does not apply: no
paths-filterusage anywhere in this diff; the touched step has no file-pattern filtering. - "Under
set -euo pipefail, a bare$(gh api …)assignment that fails kills the step with one context-free stderr line" — does not apply: the gated script still usesset -uo pipefail(no-e, pre-existing and unchanged at.github/workflows/pr.yml:356), and itsgh apicall (:386) is piped to/dev/null, not captured into a variable via$(...). - "
gh api --paginate --jqruns the filter once PER PAGE" — does not apply: the changed step'sgh apicall (:386) uses neither--paginatenor--jq.
Divergence (blind read vs PR body)
None — the body's account matches the diff on every claim graded above. The body is also appropriately conservative: its test-plan checkboxes are left unchecked ("Run IDs and statuses to be filled in") rather than claiming verification that a read-only tree (or the author) hadn't yet produced.
Reviewer checklist
- Confirm on the live PR that the second ("empty... to reproduce") commit actually produced two
pr.ymlruns on one head, that the first run'sverdictjob skipped "Set CI result," and that the head'sCI resultstatus history shows no spuriousfailurefrom the cancelled run.
There was a problem hiding this comment.
Verdict: LGTM
Claims
- VERIFIED — the verdict job's job-level
ifisalways() && (...), so it still runs when upstream jobs are cancelled/failed — matches.github/workflows/pr.yml:335(pre-existing, unchanged by this diff) - VERIFIED — "the step posts nothing" when the run is cancelled — matches the new
if: ${{ !cancelled() }}at.github/workflows/pr.yml:342 - VERIFIED — "a test leg that times out does not cancel the workflow, so the step still posts failure" — a timed-out
Run test legstep (.github/workflows/pr.yml:302-309,timeout-minutes: 25) makes thetestjob's own conclusionfailure, notcancelled; that's a different job's result flowing intoTEST_RESULT, not a cancellation signal on theverdictjob itself, socancelled()stays false and the step still runs - VERIFIED — "the dev is red step after it already requires
steps.verdict.outputs.state != '', so it skips when the verdict step is skipped" — matches.github/workflows/pr.yml:392 - VERIFIED — "the verdict step is the job's first step" — confirmed structurally, it's the first entry under the job's
steps:(.github/workflows/pr.yml:338-339) - UNVERIFIED — "a push to a draft PR... Marking the PR ready about seven seconds later starts a second run on the same head" — a live timing claim about two specific Actions runs; not checkable from a read-only tree, would need the Actions run history/timestamps for #6637
- UNVERIFIED — "Seen 2026-10-23 on #6637, run 40033237161" — a specific historical run id; would need to query that run via the Actions API/UI
Riskiest lines
.github/workflows/pr.yml:342— the new guard itself; if it over-fires it would silently swallow a genuine failure, if it under-fires the original bug (a cancelled run posting a bogusfailure) persists — UNCOVERED (no behaviour test can exercise GitHub Actions' own cancellation semantics from this repo; the PR body gives the one-sentence reason: "proven by CI runs on this PR," and the pattern is already established elsewhere in this repo —build.ymlandwpf-smoke.ymlboth useif: ${{ !cancelled() }}the same way).github/workflows/pr.yml:386— thegh api .../statusescall the new guard gates; a wrong guard surfaces here as a wrongly-posted or wrongly-withheld commit status — UNCOVERED, same reasoning as above.github/workflows/pr.yml:392— unchanged downstream step whosesteps.verdict.outputs.state != ''check is the safety net the PR body leans on if the new guard skips the step; re-read and confirmed it still holds (the output simply stays unset when the step is skipped) — not itself at risk from this diff, but it's the line that makes the change safe
Traps near this diff
- "dorny/paths-filter v4 evaluates each pattern as an INDEPENDENT predicate..." — does not apply: no
paths-filterusage anywhere in this diff; the touched step has no file-pattern filtering. - "Under
set -euo pipefail, a bare$(gh api …)assignment that fails kills the step with one context-free stderr line" — does not apply: the gated script still usesset -uo pipefail(no-e, pre-existing and unchanged at.github/workflows/pr.yml:356), and itsgh apicall (:386) is piped to/dev/null, not captured into a variable via$(...). - "
gh api --paginate --jqruns the filter once PER PAGE" — does not apply: the changed step'sgh apicall (:386) uses neither--paginatenor--jq.
Divergence (blind read vs PR body)
None — the body's account matches the diff on every claim graded above. The body is also appropriately conservative: its test-plan checkboxes are left unchecked ("Run IDs and statuses to be filled in") rather than claiming verification that a read-only tree (or the author) hadn't yet produced.
Reviewer checklist
- Confirm on the live PR that the second ("empty... to reproduce") commit actually produced two
pr.ymlruns on one head, that the first run'sverdictjob skipped "Set CI result," and that the head'sCI resultstatus history shows no spuriousfailurefrom the cancelled run.
| # request, it sets CI result and this one posts no failure. A test leg that hits its timeout also reports cancelled, but | ||
| # no newer run exists then, so that failure still posts. If the call fails, post as before. | ||
| if [ "${state}" = "failure" ] && [ "${EVENT_NAME}" = "pull_request" ]; then | ||
| newer=$(gh api "repos/${GITHUB_REPOSITORY}/actions/workflows/pr.yml/runs?event=pull_request&branch=${HEAD_REF}&per_page=20" --jq "[.workflow_runs[] | select(.id > ${GITHUB_RUN_ID})] | length" 2>/dev/null) || newer="" |
There was a problem hiding this comment.
branch=${HEAD_REF} filters GitHub's "list workflow runs" by head_branch alone. That field is the unqualified branch name — it is not qualified by head_repository/fork owner and is not the same thing as "a newer run of this workflow for this pull request." Two different contributors (especially forks) can share a branch name — GitHub's own web editor suggests patch-1, patch-2, … to every first-time contributor, so collisions on exactly that name are common for a repo that accepts outside PRs (the IS_FORK handling a few lines below shows this repo does).
Concretely: if PR A (this run) legitimately fails, and an unrelated PR B — from a different fork, branch also named patch-1 — happens to have a newer run of pr.yml, this check sees newer > 0, logs the "left to a newer run" notice, and exits 0 without setting state/failed. But run B belongs to a different PR and will set the commit status on a different SHA, never PR A's. PR A's CI result stays stuck at pending (set earlier in the plan job) with no failure ever posted and no further run to correct it — a quieter failure mode than the bug this PR is fixing, but still wrong.
The run objects returned by this same endpoint carry head_repository.full_name, which — combined with the branch name — uniquely identifies the PR. Matching on that (or on .pull_requests[].number for same-repo runs) before counting a run as "newer" would close this gap.
|
The PR title and body ("Not a fix yet... It stays a draft and must not be merged as a fix", "What this PR changes: ... That is not what HEAD ( See the inline comment on |
There was a problem hiding this comment.
Verdict: CHANGES REQUESTED
Claims
- VERIFIED — the new permission
actions: readis present and the comment explains it's for the verdict job's newer-run lookup —pr.yml:25, used atpr.yml:379 - VERIFIED — the
gh apilookup's failure is caught (2>/dev/null) || newer="") and falls through to posting as before, as the inline comment above the code claims ("If the call fails, post as before") —pr.yml:379-385; the regex guard[[ "${newer}" =~ ^[1-9][0-9]*$ ]]is false for an empty string, so the existing failure path atpr.yml:387still runs - VERIFIED — the suppression is gated to
pull_requestevents only, so the dev-push "dev is red" issue logic (pr.yml:404-465, gated ongithub.event_name == 'push') is unaffected — confirmed by re-reading that step's condition - REFUTED — PR body: "What this PR changes: ...
if: ${{ !cancelled() }}and a one-line comment above it. Nothing else." — HEAD (ec46ece6) contains noif: ${{ !cancelled() }}guard anywhere (grep -n "cancelled()" .github/workflows/pr.ymlmatches only a comment at line 374); the actual diff adds a permission, an env var, and a ~13-linegh apilookup, none of which the body mentions - REFUTED (as a description of HEAD) — PR body/title: "Not a fix yet... must not be merged as a fix" — this describes the abandoned
b56611dc7attempt's failure to reproduce; it is not an evaluation of the currentec46ece6mechanism, which the body never discusses, tests, or claims to have verified - UNVERIFIED — whether the
ec46ece6approach itself was reproduced successfully: two further "empty commit to reproduce" commits exist after it (1fdbd56a,192cdd8e), but no run IDs, statuses, or updated test-plan checkboxes are given for them
Riskiest lines
pr.yml:379—branch=${HEAD_REF}filters GitHub's workflow-run list by the unqualified branch name only, not by PR number orhead_repository, so a different PR (e.g. an unrelated fork also namedpatch-1) with a newer run id can be mistaken for "a newer run of this workflow for this pull request" — covered by no behaviour test (exercises GitHub Actions' own cross-PR run listing, not reachable from this repo's test suite) | UNCOVEREDpr.yml:380-385— the exit-0 suppression branch; if line 379 ever mismatches as above, this PR'sCI resultstays stuck at thependingset in theplanjob forever, since the run that line 379 found "newer" belongs to a different PR and will never write this SHA's status — UNCOVERED, same reasoningpr.yml:374-377— the comment's claim that a timed-out test leg "also reports cancelled, but no newer run exists then, so that failure still posts" depends entirely on the branch-scoped lookup at line 379 correctly excluding unrelated runs; if it does not (finding above), this line's stated guarantee doesn't hold — UNCOVERED
Traps near this diff
- "Under
set -euo pipefail, a bare$(gh api …)assignment that fails kills the step with one context-free stderr line... Every lookup goes through a wrapper that names the endpoint and decides whether a failure is a verdict." — does not apply: the step usesset -uo pipefail(no-e, pre-existing, unchanged), and the newgh apicall explicitly catches failure with2>/dev/null) || newer=""and falls through to the existing failure path —pr.yml:356, 379 - "
gh api --paginate --jqruns the filter once PER PAGE" — does not apply: the new call uses neither--paginatenor pagination; it's a singleper_page=20request —pr.yml:379 - "dorny/paths-filter v4 evaluates each pattern as an INDEPENDENT predicate..." — does not apply: no
paths-filterusage in this diff
Divergence (blind read vs PR body)
Substantial. The PR title and body describe and disclaim the b56611dc7 commit's if: ${{ !cancelled() }} attempt ("Not a fix yet... must not be merged as a fix"), which is no longer in the tree. HEAD is actually the later ec46ece6 commit, a different mechanism (an actions: read-gated gh api lookup for a newer run) that the body never mentions, describes, or claims to have validated. A reader relying on the PR's own narrative would not know this mechanism exists, let alone whether it works. Separately, that mechanism has the branch-matching gap in Riskiest line 1 above, which the body — being silent on the new code — obviously does not address either.
Reviewer checklist
- Update the PR title/body (or lane notes) to describe and evaluate the
ec46ece6mechanism actually at HEAD, including results of the two reproduction commits pushed after it (1fdbd56a,192cdd8e) - Reproduce the fork/branch-name-collision scenario (or confirm it can't occur under this repo's contribution model) before relying on
branch=${HEAD_REF}to scope the newer-run check to this pull request specifically - Confirm live, on an actual cancelled run, that
CI resultends upsuccess/correct rather than stuckpending
| # request, it sets CI result and this one posts no failure. A test leg that hits its timeout also reports cancelled, but | ||
| # no newer run exists then, so that failure still posts. If the call fails, post as before. | ||
| if [ "${state}" = "failure" ] && [ "${EVENT_NAME}" = "pull_request" ]; then | ||
| newer=$(gh api "repos/${GITHUB_REPOSITORY}/actions/workflows/pr.yml/runs?event=pull_request&branch=${HEAD_REF}&per_page=20" --jq "[.workflow_runs[] | select(.id > ${GITHUB_RUN_ID})] | length" 2>/dev/null) || newer="" |
There was a problem hiding this comment.
branch=${HEAD_REF} scopes this "newer run exists" lookup by branch name only, not by pull request. This repository already has the correct pattern for exactly this problem in claude-review.yml:301-306: it filters primarily on .pull_requests[].number == PR (via env.PR), falling back to branch-only matching only once a pull request closes and that field goes empty — with an explicit comment explaining why: "A run is this pull request's when its list names the number; a run with none listed... counts through the branch."
This line skips that primary filter and uses only the fallback. Two concurrently open pull requests that happen to share a head branch name — including GitHub's own auto-generated patch-1 for a one-line web-UI edit, a very plausible collision in a public repo — would make this check see an unrelated PR's/fork's newer run and conclude (wrongly) that this run's result has been "superseded." That causes state=/failed= to be reset to empty (pr.yml:382-383) and the step to exit before ever calling gh api .../statuses (pr.yml:399), so this run's actual head SHA gets no CI result status at all — not a wrong status, but a silently-missing one, left to a "newer run" that has nothing to do with this PR and will never write this SHA's status.
Suggest adding PR: ${{ github.event.pull_request.number }} to this step's env: and filtering the --jq on .pull_requests[] containing that number, the same way claude-review.yml already does, rather than relying on branch name alone.
|
The PR title and body describe and disclaim the That is not what HEAD ( See the inline comment on |
There was a problem hiding this comment.
Verdict: CHANGES REQUESTED
Claims
- VERIFIED — the comment "the verdict job asks whether a newer run of this workflow exists for the pull request (#6656)" (
pr.yml:25) — matches the lookup's actual use atpr.yml:379 - VERIFIED — the inline comment's claim "If the call fails, post as before" (
pr.yml:377) —newer=$(gh api ... 2>/dev/null) || newer=""catches a failed call, and the guard[[ "${newer}" =~ ^[1-9][0-9]*$ ]]is false for an empty or non-numeric value, so execution falls through to the pre-existing failure-posting path atpr.yml:387-392 - REFUTED — PR body: "What this PR changes:
.github/workflows/pr.yml, verdict job, step "Set CI result" (idverdict):if: ${{ !cancelled() }}and a one-line comment above it. Nothing else." — HEAD (192cdd8e, same tree asec46ece6) contains no such guard anywhere;grep -n "cancelled()" .github/workflows/pr.ymlmatches only the explanatory comment atpr.yml:374-377. The actual diff adds a newactions: readpermission (pr.yml:25), aHEAD_REFenv var (pr.yml:354), and a ~13-linegh apilookup (pr.yml:378-386) — a different mechanism added by the later commitec46ece6, which the body never mentions - REFUTED (as a description of HEAD) — title/body: "Not a fix yet... It stays a draft and must not be merged as a fix" — this evaluates the abandoned
b56611dc7attempt's failure to reproduce the fix; it says nothing about theec46ece6mechanism actually in the tree - UNVERIFIED — whether the
ec46ece6mechanism was itself validated: two further "empty commit to reproduce" commits exist after it (1fdbd56a,192cdd8e), but no run IDs, statuses, or updated test-plan checkboxes are given for either
Riskiest lines
pr.yml:379—branch=${HEAD_REF}scopes the "a newer run exists" lookup by branch name alone, not by pull request.claude-review.yml:301-306already solves this exact problem in this repo by filtering primarily on.pull_requests[].number == PR, falling back to branch-only matching only once a PR closes; this new code skips the primary filter and uses only the fallback. Two open PRs sharing a head branch name (e.g. GitHub's auto-generatedpatch-1) would make this see an unrelated PR's newer run as this one's — UNCOVERED, no behaviour test in this repo exercises GitHub's cross-PR workflow-run listingpr.yml:382-384— downstream of the above: a false match resetsstate=/failed=to empty and exits before thegh api .../statusescall (pr.yml:399), so this run's own head SHA gets noCI resultstatus at all, left to a "newer run" that belongs to a different PR and will never write this SHA's status — UNCOVEREDpr.yml:379—${HEAD_REF}is interpolated into the query string unescaped; a branch name containing#,&, or?(all valid in git ref names) can corrupt the query string or change which runs the filter matches — UNCOVERED
Traps near this diff
- "Under
set -euo pipefail, a bare$(gh api …)assignment that fails kills the step with one context-free stderr line... Every lookup goes through a wrapper that names the endpoint and decides whether a failure is a verdict." — does not apply: the script usesset -uo pipefail(no-e,pr.yml:356, pre-existing and unchanged), and the new$(gh api ...)assignment atpr.yml:379is explicitly guarded with2>/dev/null) || newer=""plus a regex check before use - "
gh api --paginate --jqruns the filter once PER PAGE: amax_byorlastis per-page, not global" — does not apply:pr.yml:379uses neither--paginatenor multiple pages, just a singleper_page=20request with a plainlengthcount - "dorny/paths-filter v4 evaluates each pattern as an INDEPENDENT predicate..." — does not apply: no
paths-filterusage anywhere in this diff - "Lite and Darling ship in parity (CONTRIBUTING's two-store rule)..." — does not apply: this is a CI workflow file with no Lite/Darling SKU counterpart requiring parity
Divergence (blind read vs PR body)
Substantial. The title and body describe and disclaim the b56611dc7 commit's if: ${{ !cancelled() }} attempt ("Not a fix yet... must not be merged as a fix"), which is no longer in the tree. HEAD actually carries the later ec46ece6 commit's different mechanism (an actions: read-gated gh api lookup for a newer run), which the body never mentions, describes, or claims to have validated — a reader relying on the PR's own narrative would not know this mechanism exists. That mechanism also has the branch-matching gap in Riskiest line 1 above, which the body — being silent on the new code — does not address.
Reviewer checklist
- Update the PR title/body to describe and evaluate the
ec46ece6mechanism actually at HEAD, including results of the two reproduction commits pushed after it (1fdbd56a,192cdd8e) - Reproduce the fork/branch-name-collision scenario (or confirm it can't occur under this repo's contribution model) before relying on
branch=${HEAD_REF}alone to scope the newer-run check to this pull request - Confirm live, on an actual cancelled run, that
CI resultends up correct rather than stuck unset
|
Follow-up pass over
One divergence worth a body update, though: the PR description's own "Considered, not added" section currently says:
|
There was a problem hiding this comment.
Verdict: CHANGES REQUESTED
Claims
- VERIFIED — commit headline "Scope the newer-run lookup to this pull request" — the
verdictjob's lookup now filters on.head_repository.full_name == env.HEAD_REPOand requires.pull_requeststo be empty or nameenv.PR_NUMBER, not branch name alone —.github/workflows/pr.yml:399-404 - VERIFIED — commit headline "read the live draft state" — the
planjob now queriesgh api repos/.../pulls/${PR_NUMBER} --jq '.draft'and overwritesIS_DRAFTwith the live value before it feedsci-plan.py --draft, instead of trusting the event payload'sdraftfield —.github/workflows/pr.yml:71-80 - REFUTED — PR body, "Considered, not added": "Filtering the runs by head repository so a fork PR with the same branch name does not count: ... No concrete trigger seen." —
d47de35a0adds exactly this filtering (pr.yml:402), reversing the stated decision; the body is unedited since and still describes the old, narrower scope as current
Riskiest lines
.github/workflows/pr.yml:402— the newhead_repository/pull_requestsscoping itself; if a newer run is wrongly excluded (an edge case in how the Actions API populateshead_repositoryorpull_requests), this run posts a real failure instead of deferring to the newer one — if wrongly included, a genuine failure is silently swallowed — UNCOVERED, no behaviour test in this repo can exercise GitHub Actions' own cross-run listing (same limitation the two earlier reviews on this PR already noted for the branch-only version of this lookup).github/workflows/pr.yml:72— the live-draftgh apicall; a failure here (rate limit, transient 5xx) falls back to the stale payload value via theelsebranch, silently reopening the exact race this commit fixes for that one run — UNCOVERED, same reasoning.github/workflows/pr.yml:76—IS_DRAFT="${live_draft}"is the only place the live value reachesci-plan.py;ci-plan-test.pyonly exercisescp.plan()'s handling of a--draftflag it's handed directly (ci-plan-test.py:132-151), so it cannot and does not cover whether the bash correctly computes that flag from the API in the first place — UNCOVERED
Traps near this diff
- "Under
set -euo pipefail, a bare$(gh api …)assignment that fails kills the step with one context-free stderr line ... Every lookup goes through a wrapper that names the endpoint and decides whether a failure is a verdict." — does not apply: the live-draft read (pr.yml:72) sits inside anifcondition underset -euo pipefail(pr.yml:66), so bash's errexit rules exempt it even on failure; the newer-run read (pr.yml:400) runs underset -uo pipefail(no-e,pr.yml:374, pre-existing) with an explicit2>/dev/null) || newer=""fallback - "
gh api --paginate --jqruns the filter once PER PAGE: amax_byorlastis per-page, not global." — does not apply: neither new call uses--paginate; both are single requests (per_page=20atpr.yml:400, a single PR lookup atpr.yml:72) - "A summary helper reading
"$1"truncates every message whoseechowraps with a trailing backslash." — does not apply: the new::notice::lines (pr.yml:74, 78) are each one physical line, no backslash continuation - "dorny/paths-filter v4 evaluates each pattern as an INDEPENDENT predicate..." — does not apply: no
paths-filterusage anywhere in this diff
Divergence (blind read vs PR body)
The body's "Considered, not added" section claims head-repository filtering was deliberately left out; d47de35a0 adds it, reversing that call with no explanation in the commit message or an updated body. The body also never mentions the live-draft-read mechanism this commit adds. Posted as a separate comment on the PR with the specifics; this is the same category of body/HEAD drift the two earlier full-pass reviews on this PR flagged for the commits before this range.
Reviewer checklist
- Update the PR body's "Considered, not added" section (or remove it) now that head-repository/PR-number scoping is implemented, and add a line describing the live-draft-read fix
- Reproduce the push-to-draft-then-
gh pr readyrace live once more against this exact commit (the PR's own reproduction table coversec46ece6/1fdbd56a/192cdd8e, notd47de35a0) and confirm the final head'sCI resultlands success, not stuck pending
| # #6656: a run that a newer push or ready-for-review replaced is cancelled, but cancelled() inside this job reports the | ||
| # job's own state, so the step cannot tell. Ask the API instead: when a newer run of this workflow exists for this pull | ||
| # request, it sets CI result and this one posts no failure. A newer run is this pull request's when it comes from this | ||
| # head repository (a fork's runs list no pull requests, so the repository is what tells two forks that both use | ||
| # `patch-1` apart) and its pull request list is empty or names this number. -f makes gh URL-encode the branch name. A test | ||
| # leg that hits its timeout also reports cancelled, but no newer run exists then, so that failure still posts. If the call | ||
| # fails, post as before. | ||
| if [ "${state}" = "failure" ] && [ "${EVENT_NAME}" = "pull_request" ]; then | ||
| newer=$(gh api -X GET "repos/${GITHUB_REPOSITORY}/actions/workflows/pr.yml/runs" -f event=pull_request -f branch="${HEAD_REF}" -f per_page=20 --jq '[.workflow_runs[] | ||
| | select(.id > (env.GITHUB_RUN_ID | tonumber) | ||
| and .head_repository.full_name == env.HEAD_REPO | ||
| and (((.pull_requests // []) | length) == 0 | ||
| or any((.pull_requests // [])[]; (.number | tostring) == env.PR_NUMBER)))] | length' 2>/dev/null) || newer="" |
There was a problem hiding this comment.
Divergence from the PR body. The body's "Considered, not added" section says:
Filtering the runs by head repository so a fork PR with the same branch name does not count: it would only matter for a fork PR sharing a branch name with another PR that has a newer run, and the result is a pending status (fail-safe), not a green. No concrete trigger seen.
But this is exactly what's shipped here: .head_repository.full_name == env.HEAD_REPO (line 402) plus the pull_requests membership check (lines 403-404) are the head-repository filter the body says was considered and left out. That filter landed in the later commit d47de35 ("Scope the newer-run lookup to this pull request and read the live draft state"), after the body's reproduction section was written, and the body was never updated to match — it still describes the pre-d47de35 version of this block.
Net effect: none of the reproduction cycles in the body (all against ec46ece6/1fdbd56a/192cdd8e) exercise this scoping filter at all. It may well be correct — the logic reads consistently (null-safe on a deleted fork's head_repository, matches only this PR's number or an empty pull_requests list) — but it's unverified by anything the body presents as evidence, and the body actively asserts the opposite of what's in the diff.
| # The payload's draft flag is as old as the event. A push to a draft and `gh pr ready` seconds apart start the push run | ||
| # last: it cancels the ready run and its payload still says draft, so it would skip and the head would get no `CI result`. | ||
| # Read the live state instead; if the call fails, keep the payload's value. | ||
| if [ "${EVENT_NAME}" = "pull_request" ]; then | ||
| if live_draft=$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" --jq '.draft' 2>/dev/null) && [[ "${live_draft}" =~ ^(true|false)$ ]]; then | ||
| if [ "${live_draft}" != "${IS_DRAFT}" ]; then | ||
| echo "::notice title=Draft state read live::The pull request is draft=${live_draft} now, but the event payload said draft=${IS_DRAFT}. Using the live state." | ||
| fi | ||
| IS_DRAFT="${live_draft}" | ||
| else | ||
| echo "::notice title=Draft state read from the payload::The pull request API call failed, so the event payload's draft=${IS_DRAFT} is used." | ||
| fi | ||
| fi |
There was a problem hiding this comment.
Undocumented in the PR body, and it directly contradicts a claim there. This live-draft re-read isn't mentioned anywhere in "What this PR changes" or the reproduction table. Worse, the body's "What I could not reproduce" section says, about the push/gh pr ready ordering race:
That is an event-ordering race between the push and ready events, not something this change touches. Worth a look in a separate issue; I did not file one.
That's the exact race this block fixes: instead of trusting the pull_request payload's stale draft flag (captured when the event fired, not when the plan job runs), it re-reads .draft live from the API so a run that lands after gh pr ready sees the current state rather than skipping as a draft. This landed in the same later commit (d47de35) as the head-repository scoping flagged in the other comment — after the body's text was written, with no update to reflect it.
No reproduction cycle in the body exercises this path (the cycles described predate the commit that added it), so the claim that the race is untouched is simply false for the code actually shipped, and the fix for it is correspondingly unverified by anything presented.
Review noteThis is a workflow-only change ( The finding is in how the PR body accounts for its own commits, posted inline on the two blocks it affects:
Both landed in the last substantive commit ( No correctness bug found in the shipped logic itself from a read-only tree; the issue is that the body's account of what was tested and what was deliberately left out no longer matches what's in the diff. |
There was a problem hiding this comment.
Verdict: CHANGES REQUESTED
Claims
- VERIFIED — "Removes round 1's
if: ${{ !cancelled() }}and its comment." — no!cancelled()line exists anywhere in the finalpr.yml(confirmed by grep across the file); consistent with it being added and removed across the PR's own commit history. - VERIFIED — "Adds
actions: readtopermissions:(the call needs it...)" — pr.yml:27. - VERIFIED — "The
dev is redstep is unchanged: it runs only on push events, where this check does not apply." — pr.yml:429 onward shows no diff to that step; itsifstays gated ongithub.event_name == 'push'. - VERIFIED (incompletely described) — "asks the API whether a newer pr.yml run exists for this branch... any run with an id above this run's" — the id check exists (pr.yml:401), but the description omits the
head_repositoryandpull_requestsscoping that the shipped filter also applies (pr.yml:402-404) — see REFUTED claim below, which is the other half of this same filter. - REFUTED — "Considered, not added: Filtering the runs by head repository so a fork PR with the same branch name does not count" — the shipped filter adds exactly this:
.head_repository.full_name == env.HEAD_REPOplus apull_requestsmembership check (pr.yml:394-404), landed in commitd47de35after the body's "Considered, not added" text was written. - REFUTED — "[the push/
gh pr readyordering race is] not something this change touches. Worth a look in a separate issue; I did not file one." — the same commitd47de35adds a live re-read of the PR's draft state in theplanjob (pr.yml:68-80) specifically to close that race (reading current state instead of the stale event payload), directly contradicting the claim that this change doesn't touch it.
Riskiest lines
- .github/workflows/pr.yml:394-404 — the
head_repository/pull_requestsscoping filter the body says wasn't added; no reproduction cycle in the body exercises it — UNCOVERED (no behaviour test exists for workflow-YAML bash in this repo; GitHub Actions logic isn't unit-tested here) - .github/workflows/pr.yml:68-80 — the live draft-state re-read, which is the actual fix for the race the body disclaims fixing; untested by anything the body presents — UNCOVERED
- .github/workflows/pr.yml:400-404 — the multi-line
--jqfilter, the first instance in this codebase of a jq program spanning physical lines inside a single-quoted argument in a file the repo's.gitattributeschecks out as CRLF (confirmed viafile .github/workflows/pr.yml); relies on jq treating the embedded\rbytes between tokens as insignificant whitespace — UNCOVERED, and unlike the other two risks this one isn't addressed by the body at all
Traps near this diff
- "Under
set -euo pipefail, a bare$(gh api …)assignment that fails kills the step with one context-free stderr line... Every lookup goes through a wrapper that names the endpoint and decides whether a failure is a verdict." (#2309) — does not apply: the plan job's live-draft read sits inside anif live_draft=$(...) && [[ ... ]]; thencondition (pr.yml:72), which bash's-edoes not trigger on; the verdict job's lookup runs underset -uo pipefailwith no-e(pr.yml:374) and falls back explicitly via|| newer=""(pr.yml:404). - "
gh api --paginate --jqruns the filter once PER PAGE: amax_byorlastis per-page, not global" — does not apply: neither newgh apicall uses--paginate(pr.yml:72, 400). - "
.gitattributessaystext=auto eol=crlf... A diff that touches every line of a file is a line-ending conversion until proven otherwise" — does not apply:gh pr diffshows isolated hunks (pr.yml:7-12, 22-27, 65-80, 367-411), not a whole-file rewrite; the file's pre-existing CRLF endings are unchanged background, not something this diff did.
Divergence (blind read vs PR body)
The PR body's "Considered, not added" and "What I could not reproduce" sections describe the pre-d47de35 version of this diff. The final commit adds exactly the two things the body says were left out or untouched: the fork/head-repository scoping, and the live-draft re-read that fixes the push/gh pr ready race. The body was written against an earlier state of the branch and never updated after the last substantive commit, so its reproduction evidence (all against ec46ece6/1fdbd56a/192cdd8e) validates none of what's actually shipped in d47de35. The shipped mechanism reads as internally consistent on a blind walk, but nothing in the body demonstrates it, and parts of the body assert the opposite of what's in the tree.
Reviewer checklist
- Confirm live that a draft PR pushed-to and then
gh pr ready'd within seconds (the exact race in cycle 1 of the body) now gets a non-pendingCI resulton its final head, now that the live-draft read and the newer-run check both ship together.
Claude posting for Erik Darling. Addressed: the body now matches the shipped filter and the plan job's live draft read, and the CRLF jq filter was run locally with CRLF and LF line endings on #6691's runs and returned the same result.
|
Non-blocking observation: |
There was a problem hiding this comment.
Verdict: LGTM
Claims
- VERIFIED — "before it posts a failure, it asks the API whether a newer run of pr.yml exists for this pull request... writes empty state= and failed= outputs and exits 0" — pr.yml:399-410
- VERIFIED — "If the call fails or returns a non-count, it posts as before." — pr.yml:404-405 (
|| newer=""plus the^[1-9][0-9]*$guard before use) - VERIFIED — "
gh api -X GET .../actions/workflows/pr.yml/runs -f event=pull_request -f branch=$HEAD_REF -f per_page=20... mirrors claude-review.yml" — pr.yml:400 and claude-review.yml:303 (-f branch="$HEAD_REF") - VERIFIED — "A run counts when its id is above GITHUB_RUN_ID, its head_repository.full_name equals this PR's head repo, and its pull_requests is empty or names this PR's number." — pr.yml:401-404
- VERIFIED — "Live draft read (plan job): reads gh api repos/$R/pulls/ --jq .draft... uses it in place of the payload's IS_DRAFT... notice on mismatch... fallback with notice on failure." — pr.yml:71-80
- VERIFIED — "pull-requests: read was already granted." — pr.yml:26 (unchanged context line)
- VERIFIED — "Adds actions: read to permissions" — pr.yml:27
- VERIFIED — "The dev is red step is unchanged: it runs only on push events." — pr.yml:430 (unchanged context,
if: ... github.event_name == 'push' ...) - UNVERIFIED — "Seen on #6637, run 40033237161" and the whole reproduction-run table (run IDs, cancellation outcomes) — live GitHub Actions run history, not inspectable from a read-only tree
- UNVERIFIED — "I ran the filter against this repo's real run list: 8 matches for this repo and PR, 0 for a different head repo." — live API measurement
- UNVERIFIED — "the coordinator ran the exact filter with CRLF and with LF line ends against the real runs of #6691... both returned 1" — live measurement; independently, the file's
.gitattributes-driven CRLF is normalized away by the YAML parser beforegh api --jqever sees the block scalar's content, so the practical conclusion holds either way
Riskiest lines
- pr.yml:399-410 — if the "newer run exists" lookup finds a run that never itself reaches
verdict(e.g. it too gets cancelled, or sits outside the 20-run/per_pagewindow), this run abstains and nothing ever setsCI result, reproducing the original stuck-pending shape for a different reason — UNCOVERED (CI workflow logic; no behaviour test in this repo's suite can exercise GitHub Actions run sequencing, and the PR body's live reproduction runs stand in for one) - pr.yml:71-80 — a misread live
.draftvalue would push a ready PR back intoskip-draftor run a draft early; the^(true|false)$regex guard makes this narrow, but it's unverifiable here — UNCOVERED, same reason as above - pr.yml:400-404 — the jq filter's
head_repository.full_name/pull_requestsmatching is the only thing stopping two different PRs that share a fork and branch name from being conflated (e.g. the same fork branch opened against bothmainanddev) — UNCOVERED, same reason as above
Traps near this diff
- "Under
set -euo pipefail, a bare$(gh api …)assignment that fails kills the step with one context-free stderr line... Every lookup goes through a wrapper that names the endpoint and decides whether a failure is a verdict." — does not apply — both newgh apicalls are guarded: pr.yml:72 sits as anifcondition (soset -edoes not fire on its failure) with an explicitelsebranch, and pr.yml:400/404 pairs the assignment with|| newer=""and a regex check before use - "dorny/paths-filter v4 evaluates each pattern as an INDEPENDENT predicate under
predicate-quantifier: some..." — does not apply —pr.ymlcontains nopaths-filterusage anywhere (grep -n "paths-filter" .github/workflows/pr.ymlis empty) - "
gh api --paginate --jqruns the filter once PER PAGE..." — does not apply — neither new call passes--paginate(pr.yml:72 fetches a single resource; pr.yml:400 fetches one page of 20) - "
.gitattributessaystext=auto eol=crlf: ... A diff that touches every line of a file is a line-ending conversion until proven otherwise" — does not apply —gh pr diffshows only the added/modified hunks, not a whole-file rewrite; the pre-existing CRLF endings are unchanged by this PR
Divergence (blind read vs PR body)
None — the body's account matches the diff on every claim graded above. (Posted a separate non-blocking comment noting that build.yml, wpf-smoke.yml and claude-review.yml have the same stale-payload draft read this PR fixes in pr.yml's plan job; the PR explicitly scopes itself to pr.yml only, so this is an observation, not a divergence.)
Reviewer checklist
- Confirm on GitHub that the final head from the PR's race-cycle test, commit
5709f554, finished withCI result=success(the PR body's own "NOT PROVEN" item at the time of writing)
Claude posting for Erik Darling.
Part of #6656.
Problem
A push to a pull request starts
pr.yml. Marking the PR ready (or pushing again) starts a second run, andcancel-in-progresscancels the first. The cancelled run'sverdictjob still runs (job-levelif: always() && ...) and its "Set CI result" step postedCI result= failure ("CI failed: plan (cancelled)") on the head. Seen on #6637, run 40033237161. A second, related defect: a push to a draft andgh pr readyseconds apart start the push run last; it cancels the ready run and skips as a draft from its stale event payload, so the final head got noCI resultat all.Why round 1 failed
Round 1 put
if: ${{ !cancelled() }}on the step. In a step'sif,cancelled()reports the job's own state, not the workflow's, so it was true in run 40033984860 and the step still posted failure. Aneeds.*.result == 'cancelled'test is wrong too: a test leg that hitstimeout-minutesis also reported as cancelled and that must still post failure.What this PR changes (
.github/workflows/pr.ymlonly)pull_requestevents, before it posts a failure, it asks the API whether a newer run ofpr.ymlexists for this pull request. If one does, it prints a::notice::, writes emptystate=andfailed=outputs and exits 0 without posting; the newer run sets the status. If the call fails or returns a non-count, it posts as before. Success is posted as before. A test leg that hitstimeout-minuteshas no newer run, so its failure still posts.gh api -X GET .../actions/workflows/pr.yml/runs -f event=pull_request -f branch="$HEAD_REF" -f per_page=20(the-fform URL-encodes a branch name with#,&or?; mirrorsclaude-review.yml). A run counts when its id is aboveGITHUB_RUN_ID, itshead_repository.full_nameequals this PR's head repo (HEAD_REPOenv var; a fork's runs list nopull_requests, so this is what separates two forks that both usepatch-1), and itspull_requestsis empty or names this PR's number (PR_NUMBER). I ran the filter against this repo's real run list: 8 matches for this repo and PR, 0 for a different head repo.ci-plan.py, onpull_requestevents, readsgh api repos/$R/pulls/<number> --jq .draftand uses it in place of the payload'sIS_DRAFT. If it differs from the payload, a notice says so; if the call fails, the payload value is used and a::notice::says so.pull-requests: readwas already granted.actions: readtopermissions:(the lookup needs it; round 3 of Part of #6656: smaller build artifact, Lite reads classes on Darling-only runs, legs fit the free runners (CI speed) #6668 had removed it with the leg cap).The
dev is redstep is unchanged: it runs only on push events.Reproduction runs (UTC, 2026-10-23/24)
Earlier lane, fix at ec46ece (no head-repo scoping, no live draft read):
skip-draftfrom the stale payload: noCI resulton the head (the race this round fixes)This round (fix at d47de35):
CI resulton 253a5cd =pending, no failure.ready_for_reviewrun of the race cycle, cancelledpendingwaiting for the concurrency group, no jobs yet when I stoppedRace cycle: PR set back to draft, empty commit 5709f55 pushed,
gh pr readyrun right after (about 2 seconds later).NOT PROVEN (runner queue)
The GitHub-hosted runners were queued for over 30 minutes on both cycles, and the old run's queued verdict job still holds the PR's concurrency group, so run 40044095342 had not started when my 60 minutes ran out. Still to confirm on the final head 5709f55: that run's plan prints the "Draft state read live" notice, sets
CI resultpending, and the result ends insuccess. Checkgh api repos/erikdarlingdata/PerformanceMonitor/commits/5709f5542/statuses. Open run IDs: 40041541464 (queued verdict), 40044095342 (pending). Update at 00:37Z: the pending run did start. Its plan job completed success andCI resulton the final head 5709f55 ispending(set by that plan), so the push-then-ready race no longer skips the head as a draft. I could not tell from the log whether the live-read notice printed, because the run was still in progress. I then rangh pr ready --undo: the PR is a draft again. Build and the test legs were still queued; the finalsuccessis not yet seen.No
actionlintwas installed locally; I checked the jq filter against the live API and the YAML edits by reading them.Bot review round 2 (changes requested)
if: !cancelled()as the change and "Not a fix yet": rewritten above to describe the API lookup at HEAD; title updated.pr.yml~379): now scoped to this PR by head repository and bypull_requests(empty or this number).-f branch=....CI resultends correct: runs 40041348424 and 40036369361 show the notice and no failure; the final success is NOT PROVEN yet (above).Considered, not added
Test plan
CI result(pending) from the push run after the ready raceCI resultendssuccess(run 40044095342 build and legs still queued)Bot review round 3 (changes requested)
.github/workflows/pr.yml400-404), described under "What this PR changes": it counts only runs from this pull request's head repository whosepull_requestslist is empty or names this pull request.gh pr readyrace is no longer described as untouched. The plan job's live draft read (pr.yml 68-80), described under "What this PR changes", re-reads the pull request's draft state from the API instead of the event payload, so a run that starts aftergh pr readysees the pull request as ready. It is meant to close that race; the final-head result is still NOT PROVEN (see above).--jqfilter (pr.yml 400-404) checked: the coordinator ran the exact filter with CRLF and with LF line ends against the real runs of Darling README: plain uninstall-darling.ps1 also refuses on an MSI install #6691 (cancelled run 40049568380, newer run 40049596679), and both returned 1, so jq treats the CR bytes as whitespace.