Skip to content

ci: rebalance portable serial test shards - #8

Merged
Profreshor merged 1 commit into
mainfrom
fm/fm-ci-serial-shard2-timeout
Oct 16, 2026
Merged

Profreshor merged 1 commit into
mainfrom
fm/fm-ci-serial-shard2-timeout

Conversation

@Profreshor

Copy link
Copy Markdown
Owner

Intent

I'll write the fork grant

Context: that was the captain's answer letting this firstmate open PRs on the captain's fork (Profreshor/firstmate) so the follow-ups from the Codex review on fork PR #3 could ship. One of them, fork PR #7 (Herdr crew scan scope), cannot go green: its "Behavior portable serial 2" job was cancelled twice at the workflow's 30-minute limit. On main the same shard took 29 minutes (run of 2026-10-16 20:41-21:10 UTC) while the other eight shards took 15-20 minutes, so shard 2 is at its limit regardless of PR 7's change.

What Changed

  • Raise the portable serial shard packing target to roughly 25 minutes and update the coverage-budget boundary checks.
  • Increase the supervision-host test timing hint so its slowest observed run packs alone in a shard.
  • Document the revised timing baseline and remaining five-minute CI-tier headroom.

Risk Assessment

✅ Low: Captain: the bounded packing-hint update isolates the measured 25-minute host suite in one serial shard while preserving the existing complete/disjoint coverage checks and 30-minute CI timeout.

Testing

Live execution of the runner produced a passing coverage report with a 1,501,963 ms maximum under the 1,510,000 ms budget and showed the slow host suite alone in shard 1. The focused contract test passed its new exact-budget and one-millisecond-over-budget checks, but later failed an unrelated PTY reparenting fixture; no remote GitHub Actions run was driven in this phase.

  • Live validation: ⚠️ inconclusive - 2 of 3 scenarios driven live against the product
Scenario Result Live Evidence
Runner reports a serial packing model below the revised budget ✅ pass live portable-serial-coverage.txt
Slow supervision-host suite is isolated in its own portable serial shard ✅ pass live supervision-host-lane.txt
Actual GitHub 30-minute shard completes after rebalancing ⏸️ untested no No authority in this test phase to dispatch or control the remote CI workflow; run the branch’s GitHub Actions portable-serial jobs to measure the real runner duration.
Evidence: Portable serial coverage report

Source: Portable serial coverage report

FM_TEST_COVERAGE ok total=263 parallel=24 parallel_max_ms=665545 parallel_imbalance_ms=2 parallel_unhinted=0 serial=222 serial_shards=9 serial_unhinted=17 serial_max_ms=1501963 serial_budget_ms=1510000 herdr=17
Evidence: Supervision-host shard assignment

Source: Supervision-host shard assignment

portable-serial-1of9
tests/fm-supervision-host.test.sh
Evidence: Focused runner contract test transcript

Source: Focused runner contract test transcript

ok - exact suite coverage: --all lists every tests/*.test.sh once
ok - family selection returns a proper subset of the suite
ok - single-script selection lists exactly that path
ok - changed-file selection stays conservative (never silent full suite)
ok - a task marker refuses execution in the primary checkout and leaves worktrees and inspection alone
ok - runner and its documentation surfaces select their curated family, not just their contract owners
ok - shell line-ending policy selects runner coverage
fm-test-run: no tests selected for changes vs HEAD (map is conservative; use --all for the complete suite)
ok - changed selection covers dependents, fails closed for live unmapped source, and accepts retired unconsumed source
ok - a bin reference selects the referencing scripts, and consumers still select their curated families
ok - changed defaults to bounded automatic scheduling with serial override
ok - Windows emulation exempts only synthetic POSIX modes
ok - a plain script list defaults to bounded automatic concurrency without an automatic timeout
ok - family proofs run concurrently only within separate family phases
ok - empty changed selection emits deterministic text and JSON summaries
ok - timing markers and JSON artifact are valid
ok - aggregate exit reflects any script failure
ok - gate-skip accounting is honest and non-failing
ok - a gate skip records why it skipped
ok - a script that actually ran records no skip reason
ok - live guards are recorded as a capability class, not a bare env opt-in
ok - fail-on-gate-skip converts herdr-not-found into a hard failure
ok - exclude-family drops the named primary family after selection
ok - proven-isolated scheduling ignores parallel hints
ok - family, all, changed, and script selections ignore parallel hints
ok - portable shard union, disjointness, and coverage guard hold
ok - portable parallel lanes are fully hinted and packed within 5% of each other
ok - portable serial shards are a deterministic disjoint cover of the serial lane
ok - coverage guard bounds the unmeasured share and serial packing within twenty-five minutes
ok - serial packing accepts the exact budget and refuses one millisecond above it
ok - portable serial shard lanes refuse mismatched, out-of-range, and countless names
ok - --jobs refuses non-proven / stateful selections
not ok - --jobs on a proven family must be admitted, got 1:  FM_TEST_BEGIN 2026-10-16T22:48:29Z tests/fm-session-lock-ancestry.test.sh family=watcher-wake-lock expected_gate_skip=none
FM_TEST_BEGIN 2026-10-16T22:48:29Z tests/fm-supervision-events.test.sh family=watcher-wake-lock expected_gate_skip=none
ok - handle_push_transition: a blocked crew enqueues a stale wake naming its window and wakes the supervisor
ok - handle_push_transition: enqueue failure cannot commit the Herdr dedupe marker
ok - handle_push_transition: a declared-pause crew is absorbed (no fast wake), left to the poll loop's long cadence
ok - handle_push_transition: a captain-held crew is absorbed (no fast wake), left to the poll loop's long cadence
ok - event_wait_or_sleep: herdr windows go on the event pane list, but kind=secondmate endpoints are excluded
ok - event_wait_or_sleep: one cached capability probe owns validation across bounded waits
ok - event_wait_or_sleep: a home with no push-capable window is inert (sleeps POLL, never touches the event path)
ok - event_wait_or_sleep: consecutive event-path failures disable the fast-path and revert to pure polling (fail-closed)
# fm-supervision-events.test.sh: all assertions passed
FM_TEST_END 2026-10-16T22:48:29Z tests/fm-supervision-events.test.sh exit=0 duration_ms=495 gate_skip=false
ok - session-lock: a version-named Claude Code session is identified from its install path and argv[0]
ok - session-lock: a harness that is pid 1 of its own namespace is examined, not skipped
ok - session-lock: ordinary script paths under a harness directory are not harness processes
ok - session-lock: ownership stops at the first non-harness gap above the contiguous run
ok - session-lock: a live version-named session holding the lock is not mistaken for a stale owner
ok - session-lock: a trusted same-session id keeps owning a recycled background chain, and nothing weaker does
ok - session-lock: a trusted id anchors the lock on the model-loop process, anything else on the outermost pid
ok - session-lock e2e: a version-named session claims the home and arms supervision
ok - session-lock e2e: a session parented by a harness-named daemon claims the home and arms supervision
ok - session-lock e2e: a version-named session under a harness-named daemon keeps its own lock
not ok - the pty-host was not reparented to init after the daemon ended
FM_TEST_END 2026-10-16T22:49:37Z tests/fm-session-lock-ancestry.test.sh exit=1 duration_ms=67666 gate_skip=false
FM_TEST_SUMMARY total=2 failed=1 skipped_gate=0 duration_ms=67774
FM_TEST_SUMMARY_FAMILY family=watcher-wake-lock count=2 duration_ms=68161 failed=1
FM_TEST_SLOWEST rank=1 script=tests/fm-session-lock-ancestry.test.sh duration_ms=67666
FM_TEST_SLOWEST rank=2 script=tests/fm-supervision-events.test.sh duration_ms=495
- Outcome: ⚠️ 2 warnings across 1 run (7m38s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

⚠️ **Test** - 2 warnings
  • ⚠️ tests/fm-test-run.test.sh - The focused runner contract test reached and passed the changed packing scenarios, then failed an unrelated session-lock fixture because its fake PTY host was not reparented to init after daemon exit. This reproduced on the captured rerun and requires a host/process-namespace environment where orphaned fixture children are adopted as the test expects.
  • ⚠️ live validation verdict: inconclusive (2 of 3 scenarios were driven live against the product); untested: Actual GitHub 30-minute shard completes after rebalancing
  • Live validation: ⚠️ inconclusive - 2 of 3 scenarios driven live against the product
Scenario Result Live Evidence
Runner reports a serial packing model below the revised budget ✅ pass live portable-serial-coverage.txt
Slow supervision-host suite is isolated in its own portable serial shard ✅ pass live supervision-host-lane.txt
Actual GitHub 30-minute shard completes after rebalancing ⏸️ untested no No authority in this test phase to dispatch or control the remote CI workflow; run the branch’s GitHub Actions portable-serial jobs to measure the real runner duration.
  • bin/fm-test-run.sh --check-coverage
  • Enumerated bin/fm-test-run.sh --list --lane portable-serial-{1..9}of9 to locate fm-supervision-host.test.sh
  • tests/fm-test-run.test.sh (focused runner contract; changed packing assertions passed, later unrelated PTY reparenting fixture failed)
  • git diff ba356f1cd9002b3c8432ac8a409baf216f2697c5 | jev find "a change unrelated to raising the portable serial modeled budget and isolating the slow supervision host suite" (no unrelated hunk flagged)
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-16T23:02:52.725412Z aebe736 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@Profreshor
Profreshor merged commit 4a664eb into main Oct 16, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant