Repository navigation
Process runner: stop the owned group when the caller is terminated - #15
Merged
Merged
Conversation
run() killed its group on timeout or interruption only from an exception handler, which never runs when the calling Python process receives SIGTERM or SIGHUP (outer timeout, cancelled job, closed terminal), leaving descendants under init. Map both signals to SystemExit while waiting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NJQdbJCWyxx5EULVg4ScX3
florian6973
marked this pull request as ready for review
November 5, 2026 03:33
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
runtime.process.run(used by the MEDS-DEV conformance helpers and orchestration) starts its command in a newprocess group and kills the group from an
except BaseExceptionhandler on timeout or interruption. If thecalling Python process is terminated by SIGTERM or SIGHUP, the handler never runs: Python exits on these
signals without unwinding, so the whole group is left running. Typical causes are an outer
timeoutaroundpytest, a scheduler cancelling the job, or a closed terminal.
Found while running GenHPF MEDS-DEV gates. An outer
timeout 5400(the same value asRUN_TIMEOUT) anddetached background shells terminated pytest. That left three
meds-dev-modeltrees, each holding about 4cores in upstream training, running for 6–9 hours under init on a shared machine.
Change
runtime/process.py: whilerun()waits, SIGTERM and SIGHUP raiseSystemExit(128 + signum), so theexisting cleanup kills the owned group before the caller exits with the conventional status. This applies only
on POSIX, only in the main thread, and only where the handler is still the default (an inherited
nohupignoreis kept). Previous handlers are restored afterwards. SIGKILL of the caller still cannot be handled.
tests/test_process.py:test_terminating_the_caller_stops_descendant_work[SIGTERM|SIGHUP]signals a processblocked in
run()and checks that it exits with128 + signumand that its grandchild does not keep working.Tests
uv run pytest -q: 153 passed. With the unchangedprocess.pyfrommain, both new tests fail(2 failed, 3 passed in
tests/test_process.py).ruff check .andruff format --check .: clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01NJQdbJCWyxx5EULVg4ScX3