Skip to content

test(orchestrator): record what the SIGKILL resume test observes - #5

Merged
akynte merged 1 commit into
mainfrom
test/crash-replay-transcript
Oct 13, 2026
Merged

akynte merged 1 commit into
mainfrom
test/crash-replay-transcript

Conversation

@akynte

@akynte akynte commented Oct 13, 2026

Copy link
Copy Markdown
Owner

TestSIGKILLResume now writes down each step it checks and compares the sequence with internal/orchestrator/testdata/crash-replay.json. Any difference fails the test, so a passing CI run proves the file is what a real kill and resume produced.

Steps recorded (each is a value the test reads, not a description):

  • the task's state after creation;
  • runner A, in its own process, has written a file and is mid-turn;
  • SIGKILL, and what Wait returns;
  • the state the dead runner left: lease owner, phase, attempt;
  • runner B's refusal while the lease is fresh;
  • the stale-lease timeout (50 ms in the test, 2 m in production);
  • what runner C's agent is told on resume, and whether the killed attempt's file survived;
  • the reconciled attempt, the task's event log, and the final state.

Task ids and wall-clock times are normalised, so the file is identical on every run (checked 10 times locally, and under -race). -update-replay rewrites it when behaviour changes on purpose.

Test-only change: no production code is touched. go vet ./... and go test -race ./... pass.

TestSIGKILLResume now writes down each step it checks (the dead runner's
state, the refused second runner, the stale-lease takeover, the reconciled
attempt, the task's event log and the final state) and compares the
sequence with testdata/crash-replay.json. Any difference fails the test,
so a passing CI run proves the file is what a real kill and resume
produced; -update-replay rewrites it when behaviour changes on purpose.
Task ids and wall-clock times are normalised so the file is identical
on every run.

This gives readers (and the portfolio's replay of the test) a transcript
that cannot drift from the code.

Signed-off-by: akynte <104795207+akynte@users.noreply.github.com>
@akynte
akynte merged commit 4f2ec05 into main Oct 13, 2026
13 checks passed
@akynte
akynte deleted the test/crash-replay-transcript branch October 13, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant