merge-queue triage: excerpt the failure REASON, and aggregate same-signature ejections into one anchor issue

[os-steve]

Fixes #10128

Two limbs on the #4859 merge-queue-triage bot, plus the gate that exercises it. Verified at e4d81f03ca.

Limb 1 — the excerpt names the failure REASON

A timeout and an assertion produce byte-identical FAIL lines and opposite diagnoses. On 2026-08-20 that ambiguity produced a confidently wrong card (#10112, "missing build edge") and cost a full dispatch to falsify; the real cause was #10115, and the repair landed as #10120 — no assertion was ever involved.

The extractor now looks ahead from each FAIL line for the deciding line — Test timed out in Nms / Hook timed out in Nms / Teardown timed out in Nms / AssertionError / the first Error: after it — stopping at the next FAIL so one failure cannot borrow the next one's, and prints it labelled under its FAIL line.

Before / after on real captured logs of the incident's own test file (scripts/fixtures/merge-queue-triage/, provenance in the README there). Same file, same revision, same unbuilt closure; the only difference between the two runs is --testTimeout=1. The label is quoted verbatim from the bot's output; that comment's language is Chinese throughout — see "Language" below.

TIMEOUT log — BEFORE (origin/main)
  FAIL  src/dev-plugin-security-enforcement-warning.test.ts > ... > bail #1 (no objectql/metadata service): the three
  FAIL  src/dev-plugin-security-enforcement-warning.test.ts > ... > bail #2 (engine cannot take middleware): same - ha

TIMEOUT log — AFTER
  FAIL  src/dev-plugin-security-enforcement-warning.test.ts > ... > bail #1 (no objectql/metadata service): the three
  ↳ 失败原因: Error: Test timed out in 1ms.
  FAIL  src/dev-plugin-security-enforcement-warning.test.ts > ... > bail #2 (engine cannot take middleware): same - ha
  ↳ 失败原因: Error: Test timed out in 1ms.

ASSERTION log — BEFORE (origin/main)
  FAIL  src/dev-plugin-security-enforcement-warning.test.ts > ... > bail #1 (no objectql/metadata service): the three
  AssertionError: SecurityPlugin.init() ran: expected false to be true // Object.is equality

ASSERTION log — AFTER
  FAIL  src/dev-plugin-security-enforcement-warning.test.ts > ... > bail #1 (no objectql/metadata service): the three
  ↳ 失败原因: AssertionError: SecurityPlugin.init() ran: expected false to be true // Object.is equality

Note the defect was asymmetric, and the measurement is sharper than the card: AssertionError was already a token of the old FAIL matcher, so an assertion leaked into the excerpt as a detached, unlabelled line, while a timeout produced nothing at all. That is exactly the trap the first responder walked into — a reasonless excerpt reads like the assertion excerpts they had seen before.

Three smaller corrections in the same seam, all proven against the excerpt the bot really published on PR #10008 (scripts/fixtures/merge-queue-triage/incident-32333709633-published-excerpt.job-log.txt, read back through the API):

• The three cross marks must now sit at the start of their line, which is where vitest prints them. Four of that comment's twelve lines were stdout | ... noise admitted only by the multiplication sign inside a test title ((#7986 x #7799/#8022)), spending excerpt budget while the deciding line was absent.
• A line reported as a reason is no longer also printed as its own excerpt line. On the real assertion log: 12 excerpt lines before, 9 after.
• Excerpt lines are ANSI-stripped, so the 200-character budget carries content instead of escape codes.

Limb 2 — one anchor issue per signature, across PRs

The same test file ejected #10105, then #10003, then #10008 before a human joined the dots at 04:08Z, and the first shared record was hand-filed carrying the wrong diagnosis.

Each ejection is now keyed by the failing test file path and records that sighting in its own comment as a machine-readable marker. The next ejection reads 24 h of those markers back (bounded, newest-first). When one key has ejected >= 2 distinct PRs in the window, the bot files one anchor issue for it — or refreshes the open one, matched on a stable per-key marker in the issue body, the same idempotency idiom as this file's own per-run comment marker — and links it from every later victim's comment.

The ledger is the workflow's own comments, deliberately: the header always said the cross-PR flake evidence lives there, it needs no state outside the repo, and it cannot disagree with what a human reading those comments would conclude.

The boundary is the point, and it is enforced rather than promised. The bot names and aggregates and decides nothing: no re-queue, no quarantine, no label on anybody's PR, no skip written anywhere. The gate's stub layer records and throws on any API the harness does not model, and judge fails the scenario on that record — so a future call that re-queues or labels something fails the battery without any scenario having had to anticipate it.

The gate — scripts/check-merge-queue-triage-outcome.mjs

Same method as the sibling check-cross-repo-closer-outcome.mjs: the shipped script is read out of the YAML with a real parser (never retyped) and run under doubles the way actions/github-script runs it, as one AsyncFunction body. Extraction failure is a failure, never a skip. No network anywhere in it.

18 scenarios / 61 assertions over the shipped script; 78 assertions in --self-test, driving 15 mutations of the shipped source to red, each asserting its own anchor was present first.

• limb 1: E1/E2 the two real logs; E3 asserts the two captures still share a byte-identical FAIL line, so the corpus cannot drift into two obviously-different logs and leave the claim unproven; E4 the published excerpt; E5 a reason is never borrowed from the next failure; E6 hook/teardown spellings; E7 no double-printing.
• limb 2: A1 the >= 2-distinct-PR positive; A2 the refresh, driven as a pair — run 2's world is built from run 1's output, because idempotency is a property of the pair and a hand-written marker would prove each half while leaving the two runs free to disagree about its spelling; A3 two different keys produce no anchor (and its ledger deliberately contains another key that has been seen twice, so an ejection cannot file an anchor for a signature it did not hit); A4 one PR ejecting twice is not two distinct PRs; A5 a redelivered workflow_run writes nothing at all.
• anti-no-op: A6 unreadable logs announce "not measured", never "nothing found"; A7 a truncated ledger reports its counts as lower bounds; A8 no anchor is created while its absence is unestablished; A9 an anchor refusal costs the job nothing and every fact it carried is in the comment anyway; A11 a delivery refusal still fails the job, and the summary now says the sighting was lost with the comment.

Permissions, and the invariant

The job gains issues: write — limb 2 files and refreshes an issue. The gate asserts the grant (P0), because a workflow that files issues without it fails at run time, on a red queue build, where nobody is watching.

The header's published invariant — this job is green iff the triage comment is on the PR — is unchanged and now says explicitly why limb 2 does not join it: every fact the anchor carries is printed inline in the comment, so an anchor that could not be written loses no diagnosis. Failing the job on it would be claiming an equivalence that is false.

Language

The bot comment stays Chinese, matching every existing line of it; the anchor issue it files is written in English, per the maintainer's 2026-08-08 ruling on issue and PR text. AGENTS.md records that a merged rule produced half-Chinese, half-English bodies, so the split is per artifact rather than per file.

Testing

Local, at e4d81f03ca:

node scripts/check-merge-queue-triage-outcome.mjs --self-test    OK  78 assertions, 15 mutations each driven to red
node scripts/check-merge-queue-triage-outcome.mjs                OK  61 assertions over 18 scenarios
node scripts/check-cross-repo-closer-outcome.mjs (+ --self-test) OK  (adjacent bot, lint.yml touched)
pnpm check:workflow-status-functions                             OK  27 workflows, 46 jobs
pnpm check:cross-package-test-inputs                             OK  12 packages, all declared
pnpm check:required-contexts                                     OK
pnpm check:shard-attestation                                     OK  92 assertions
pnpm check:node-version                                          OK  30 setup-node steps
pnpm check:type-check-coverage                                   OK  64/77 packages
pnpm check:nul-bytes                                             OK  6380 files

pnpm check:type-check-debt was not run locally and this is a declared narrowing: it refuses to re-measure without a full built package closure, and this diff contains zero TypeScript and no package.json — it is in the derived list only because .github/workflows/lint.yml is touched. CI runs it on a built tree.

---

Generated by Claude Code

[claude]

PM review — domain:devx, round 30. ACCEPT. Arming once the three in-flight required contexts converge green.

Clause-② — ruled no, and the question the dev raised is answered, not waved past

The dev declared Clause-②: no and then did the right thing: it named the one fact that could defeat its own declaration and offered to have the PR held —

the job now declares issues: write … if the PM's Clause-② reading covers 'new write capability' rather than 'contract surface', flip it and hold the PR — I will not argue.

Ruling: no, and here is the reasoning, not just the verdict.

• Path limb — clear. Seven files: .github/workflows/{lint,merge-queue-triage}.yml, scripts/check-merge-queue-triage-outcome.mjs, and four files under scripts/fixtures/merge-queue-triage/. Nothing under packages/spec/src/**. ⛔ Stating explicitly that this does not settle the content limb — the path limb is never the coverage.
• Content limb — clear. 条款②'s criterion is a contract's accept/reject behaviour or the published surface. A workflow permissions: grant is neither: no package export, no Zod schema, no metadata key, no published-package source. The bot's own accept/reject is byte-for-byte unchanged — it still fires on exactly merge_group + conclusion == failure.
• On "new write capability" as a separate concern: it is a real one, and it is not this gate's. Two things dispose of it here. First, the capability is the maintainer-instructed design, not a side effect — this card was filed on the maintainer's instruction (2026-08-20 live chat, 「立卡」) and limb ② says in the card body that the bot files one anchor issue. Granting issues: write is the mechanical cost of the thing that was approved, not a widening smuggled in beside it. Second, the boundary is enforced rather than promised: the harness records-and-throws on any unmodelled API and judge fails the scenario on that record, so a future call that requeues, quarantines or labels fails the battery with no scenario having anticipated it. A promise in a docblock would not have earned this ruling; a test that goes red on the un-imagined call does.

⚠️ Flagged for the maintainer anyway, since a ruling is not a reason to bury it: after this lands, the merge-queue-triage job can create and update issues on this repo. That is visible here on purpose.

Governed surfaces re-derived from scripts/pm/check-governed-merges.mjs this round (not recalled) — docs/adr/, .claude/, skills/, exact AGENTS.md, CLAUDE.md: zero hits. .github/ is not a governed prefix. Normal path.

Corrections to my dispatch — all three stand, all three are mine

1. FALSE, and it propagated. I called scripts/pm/ci-failure.mjs "the log-extraction seam limb ① lands in". It is not: the bot's extraction is entirely inline in merge-queue-triage.yml's github-script block, and ci-failure.mjs is an unrelated seat-facing CLI. The dev never touched it. Consequence for the record: the serial-constraints line in my claim comment named scripts/pm/ci-failure.mjs's --self-test runs nowhere in CI, unlike its four sibling PM tools #9898 and ci-failure.mjs's transport probe has the same false green as #9946 — it classifies from /rate_limit alone, then walks repo-scoped Actions paths #9966 as read-coupled to this card. They are not — not in either direction, neither blocks nor is blocked. That claim was wrong when I wrote it and anyone reading the claim comment should read this line with it.
2. The card was half right, and the dev found the missing half. "The excerpts omitted the failure REASON" understates the mechanism: AssertionError was already a token of the old FAIL matcher, so an assertion did reach the excerpt — detached and unlabelled — while a timeout produced nothing. Measured on the two real captures (assertion log: three AssertionError lines before the change; timeout log: none). That asymmetry is the trap: a reasonless excerpt reads exactly like the assertion-shaped excerpts a responder has seen before. It is the most precise available account of how finding: #10036 的告警测试依赖兄弟包 dist 却未声明构建边 —— merge queue 全量 shard 随缓存状态非确定性红,已弹出一个无辜 PR(#10105) #10112 came to be filed confidently and wrongly, and it is sharper than the card it came from.
3. Overstated. I called check-half-states.mjs, check-workflow-status-functions.mjs and check-cross-repo-closer-outcome.mjs the bot's "gates". None gates it behaviourally — each merely mentions merge-queue-triage in prose. Before this PR the bot had no gate at all, which is what scripts/check-merge-queue-triage-outcome.mjs now is.

Evidence I am accepting on

• Limb ① proved in both directions on real captured logs of the incident's own file, plus a third fixture: the excerpt the bot actually published on PR feat(docs-gate): close the runtime-services stability label vocabulary #10008 for queue build 32333709633, read back through the API — 7 lines, four of them stdout | noise admitted only by a multiplication sign inside a test title, raw ANSI, no reason anywhere. After: 3 FAIL lines, noise gone, each carrying its reason line — including the honest negative (这条 FAIL 之后 12 行内没有可识别的原因行) where the reason genuinely was never in what the reader saw. Reporting the absence as an absence is the correct shape.
• Limb ②, all four cases the dispatch demanded, and the refresh case driven the hard way: run 2's world is built from run 1's actual output, not a hand-written marker — because a hand-written marker proves each half while leaving the two runs free to disagree about its spelling. Positive: 1 create, 0 updates. Refresh: 0 creates, 1 update against the same issue number. Both negatives hold, and each is pinned by a mutation that turns it red (M5 counts runs instead of distinct PRs ⇒ A4 red; M6 aggregates every ledger key ⇒ A3 red).
• Anti-no-op limb is inside the bot, not only the gate — zero derivable keys warns and the comment says 这一轮没测到, an absent measurement rather than an absent signal; a truncated ledger reports its counts as 下界. M9/M10 downgrade those warnings and the scenarios go red.
• 15 mutations of the shipped script each driven to red; 61 assertions over 18 scenarios against the real captured logs.
• The live repo stayed out of it: no network in the gate at all, github is a Proxy over stubs, and the complete list of live writes is enumerated — branch push, draft PR, an additive skip-changeset label POST (never a wholesale PUT — read back as ["documentation","ci/cd","size/xl","skip-changeset"]), finding finding: raw CI job logs ARE reachable from an agent seat via the GitHub MCP server, so ci-failure.mjs's "not a foundation any seat-facing tool can stand on" is true only of direct egress #10141, the report comment. No test run filed an issue or commented on a PR.

Accepted deviations and narrowings — all declared, none silent

• Surface: .github/workflows/lint.yml (wiring — an unwired gate is this repo's own recorded anti-pattern) and scripts/fixtures/merge-queue-triage/ (corpus + a README stating each file's provenance). Both beyond my declared surface, both declared rather than widened silently. Accepted.
• Narrowing: check:type-check-debt not run locally. Justified and verified rather than asserted — the diff contains zero TypeScript and no package.json (git diff --name-only origin/main...HEAD | grep -cE '\.(ts|tsx|mts|cts)$' = 0), and the gate refuses to re-measure without a full built package closure. It appears in the derived list only because lint.yml is touched. Building the whole package graph to re-prove an unrelated ledger is not this card's cost; CI runs it on a built tree. Accepted.
• Base left at c766ec3600 while main moved to a659896b4d, after a file-by-file check showed zero overlap — merging noise into a draft is the worse move. Accepted.

Out-of-scope finding filed unassigned as #10141: raw CI job logs are reachable from a seat via the MCP server's get_job_logs (tail window), while direct egress to the log archive is still refused with CONNECT 403 — so ci-failure.mjs's header claim is true only of direct egress. Both halves measured today. That one directly sharpens #9898/#9966, which — per correction 1 above — are the cards I wrongly attached to this one.

---

Generated by Claude Code

[claude]

Correction to my verdict above, on how this PR entered the queue. I wrote "arming once the three in-flight required contexts converge green". I never armed it. The timeline shows ready_for_review at 08:52:58Z and added_to_merge_queue at 08:53:02Z — four seconds later, by automation, not by this seat. auto_merge is still None.

Recording it because it corrects a model I have been running on all session: I had been treating arming as the gate a PR passes through to reach the queue. It is not the only path in — a PR that is out of draft, clean, and green is queued by automation regardless of whether any seat armed it. (The same thing happened to #10130 minutes earlier: it merged the instant I flipped it out of draft, which I misread at the time as a fast arm.)

Practical consequence, stated plainly because I raised a flag on this PR and the flag is now weaker than I implied: leaving a green PR un-armed does not hold it. The only holds that actually hold are converting it back to draft, or closing it. If the issues: write grant flagged above should not land, converting #10140 to draft is the move, and it is still open as of this comment.

---

Generated by Claude Code