Skip to content

merge-queue-triage bot: excerpt the failure REASON line, and aggregate same-signature ejections across PRs into one anchor issue (2026-08-20 queue-incident layer ③) #10128

Description

@os-zhuang

Filed by the spec seat on the maintainer's instruction (2026-08-20 live chat, verbatim: 「立卡」, accepting layer ③ — ranked highest-leverage of the four: it shortens the diagnosis loop for the NEXT unknown incident, whatever its cause). Provenance: the #4859 triage workflow this extends; incident thread #10115 · #10112 · PR #10120.

The two diagnosis failures this automates away, both measured tonight

  1. The excerpts omitted the failure REASON. The bot's log extraction showed FAIL src/dev-plugin-security-enforcement-warning.test.ts (and on earlier ejections, stale-looking assertion lines) but not the deciding line — Test timed out in 5000ms vs AssertionError: …. A first responder locally reproduced a DIFFERENT failure of the same file (unbuilt closure ⇒ assertion) that matched the excerpt's shape, and filed a confidently wrong diagnosis (finding: #10036 的告警测试依赖兄弟包 dist 却未声明构建边 —— merge queue 全量 shard 随缓存状态非确定性红,已弹出一个无辜 PR(#10105) #10112, "missing build edge"). One full dispatch was spent falsifying it — the actual CI logs contained zero assertion failures and only timeouts, a distinction one grep in the bot would have surfaced.
  2. Nobody connects ejections across PRs automatically. The same test file ejected three unrelated PRs (docs(skills): triage fire opens with by-name tool loading — stop the 55-schema ToolSearch ticket on self-retire rounds #10105docs(os-dev): an ablation must prove its mutation landed on disk #10003feat(docs-gate): close the runtime-services stability label vocabulary #10008) before a human joined the dots at 04:08Z. Each victim PR got its own bot comment; no shared record existed until hand-filed — and the hand-filed one mis-diagnosed (see 1).

The change, two limbs on the existing workflow (scripts/ + the #4859 workflow)

  1. Reason line in every excerpt: alongside the FAIL line, grep the job log for the matching failure-reason line (Test timed out in … / AssertionError / Hook timed out in … / first Error: after the FAIL) and print it labeled. One extraction addition; the excerpt becomes self-triaging for the timeout-vs-assertion split that burned tonight.
  2. Cross-PR signature aggregation: key = failing test file path (per queue build). When the same key appears in ejections of ≥2 distinct PRs within a rolling 24h, the bot files ONE anchor issue (or refreshes the existing open one — dedup by a stable marker in the issue body, the same idempotency idiom the bot's own <!-- merge-queue-triage:… --> markers use) listing every affected PR/build, and links it from each subsequent victim's comment. Single anchor = single place for the fix conversation, and the victim count is visible instead of re-derived.

Boundary, explicit: the bot still decides nothing — it names and aggregates; humans and seats still triage. No auto-requeue, no auto-quarantine (gate-weakening stays a human act).

Dedup: no existing card extends the triage bot (searched: merge-queue-triage / signature aggregation / failure reason). The bot's own workflow #4859 is its anchor and stays open for its own scope.

Suggested size/model: M, opus.

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions