⚡ Remeasure test weights at ed5369fc and recalibrate the runtime shards

[taras]

Why

Every runtime's shards run over the 300 s ceiling specs/testing-spec.md sets. On main at 392c006 (run 33574538193, the one that opened #714) the worst Test steps were Deno 399 s, Node 388 s and Bun 491 s. The committed weights dated from 170a021, fifty commits back: 43 Deno, 32 Node and 31 Bun files had no weight and were charged the corpus maximum, so test-deno (6/9) predicted 587 s for itself and the split balanced nothing real.

What changes

Before: test-weights.json measured at 170a021; counts 9/5/2; predicted per-shard totals 587 s (Deno), 634 s (Node), 909 s (Bun) with the fallback files.

After: test-weights.json is byte-for-byte the artifact of measure-test-weights run 33576103883, dispatched at ed5369fc on ubuntu-latest (deno 2.9.5 / node v22.23.2 / bun 1.3.14; provenance in the file's source block). Every applicable file in all three runtimes is measured; the fallback set is empty. The corpus grew: serial sums Deno 2596 s (370 files, was 1939 s), Node 1118 s (286, was 760 s), Bun 715 s (285, was 429 s). The measured lower bounds are 9/4/3, and ci.yml installs them as the calibration's starting counts, predicting 289/280/238 s per shard.

Calibration

Per the spec's rule: five consecutive runs on one fixed head keep every shard Test step and each runtime's execution window under 300 s; a miss increments only that runtime and starts a fresh five. #606 measured runners executing 15–50% above the isolated per-file weights, so the Deno floor is expected to miss; the table below records each head as it runs.

head	counts	run/attempt	result
1ea893b8	9/4/3 (floors)	33581198496 att.1	Deno MISS (worst 357s, window 373s); Node MISS (worst 452s, window 459s); Bun MISS (worst 282s, window 338s)
1981e9b1	10/5/4	33581898672 att.1	Deno MISS (worst 323s, window 332s); Node MISS (worst 334s, window 345s); Bun PASS (worst 225s, window 226s)
5bc7e0ad	11/6/4	33582364710 att.1	Deno steps all under (worst 298s) but window 409s — see below; Node MISS (worst 301s, window 341s); Bun PASS (worst 218s, window 226s)
87467062	11/7/4	33582959074 att.1	Deno MISS (shard 4/11 at 323s, window 330s); Node PASS (worst 251s, window 262s); Bun PASS (worst 223s, window 225s)
77d4cbc6	12/7/4	33583487112 att.1–3	att.1 all PASS (Deno 261s/270s, Node 253s/267s, Bun 222s/224s); att.2 all PASS (Deno 289s/298s, Node 260s/266s, Bun 225s/228s); att.3 Deno MISS (shard 7/12 at 296s, window 301s), Node steps under (255s) but window 306s while #718's CI ran alongside (queueing), Bun PASS (237s/285s); att.4 cancelled once the head moved
04ca08bf	13/7/4	33584864007 att.1–3	all PASS three times: Deno 256s/275s, 243s/249s, 262s/268s; Node 256s/285s, 259s/263s, 256s/259s; Bun 231s/247s, 236s/236s, 235s/239s — sequence stopped here because main moved (below)

main advanced to a7f60c0 during the sequence: #718 added packages/cli/tests/stdout-delivery.test.ts (a corpus change — an unmeasured file is charged the Deno maximum, 97 s, so the partition on a rebased head would differ) and the workflows moved Bun from 1.3.14 to 1.4.0. Both are remeasure triggers, so a second measurement was dispatched at a7f60c0 (run 33585991716); the branch will be rebased, carry that artifact, and run its five on the rebased head.

The window clause measures the account's runner queue as well as the partition. Run 33582364710 overlapped #718's CI run (02:09–02:18 against 02:12–02:20); the account runs about 20 jobs at once and the two runs held 33 and 30, so a third of this run's shards waited for a slot — test-deno (5/11)'s Test step began 124 s after test-deno (2/11)'s, and Deno's 409 s window was that skew on a 285 s test with every step under 300 s. The next run had the account to itself and every window sat 2–11 s over its worst step, as #606's did. So the clause is satisfiable, but a fixed head's five consecutive runs can lose one to another branch pushing at the same time, and that miss says nothing about the count. Node's 301 s step at 6 and Deno's 323 s step at 11 were genuine misses and incremented.

Runner overhead over the isolated weights, by runtime, across these runs: Deno +24–37%, Bun +23–26%, Node +49–62%.

How to verify it

• scripts/tests/ci-workflow.test.ts "the actual partition the workflow installs" proves the committed corpus, exclusions, weights and ci.yml counts form a complete, disjoint split with no empty shard — the count-sensitive test, so ci.yml stays the single count encoding.
• Focused partition evidence at 1ea893b — ci-workflow, test-shards, test-weights, runtime-tests, shard-execution, measure-test-weights, runtime-exclusions, test-file-discovery: ok | 27 passed (116 steps) | 0 failed.
• Each shard logs its predicted total and its unmeasured files before it runs; every shard on this PR should log 0 unmeasured.

Scope

Included

• test-weights.json, byte-for-byte from the measurement artifact (sha256 858f27d1…d4604).
• ci.yml shard counts, calibrated by the five-run rule.

Intentionally unchanged

• Runner commands, corpus, exclusions and in-shard concurrency — a miss is answered with a count, never with these.
• The plan.test.ts flake that actually reddened main is 🐛 Keep the session-name assertion off the random profile root (#714) #716.

Generated or mechanical changes

• test-weights.json comes from .github/workflows/measure-test-weights.yml; no millisecond was edited by hand.

Scope confirmation

• Every changed file supports the purpose described above.
• Unrelated cleanup and formatting changes are excluded.
• Generated or mechanical changes are clearly identified.
• The description matches the final diff and test results.

[github-actions]

PR #721: ⚡ Remeasure test weights at ed5369f and recalibrate the runtime shards

2 files, +939 / -833

Scope

🔴 PR has 1772 lines changed. Split into focused PRs.

🟡 1772 lines changed. PRs under 400 receive more thorough review.

Structural

✅ No structural bloat detected.

Slop

✅ Slop indicators look low.

Static Analysis

✅ Oxlint found no issues.

Correctness

No extraneous code patterns detected.