Repository navigation
test(runner): generate the nextest profile from shared timeout constants - #5450
Conversation
…rows Program rows relied on the program's 900 s default, so one hung test could use much of the 45-minute run timeout. Derive the per-test timeout from the same constants as the nextest profile (120 s) and pass it with a 10 s grace to the program. Part of #5416. Signed-off-by: Evan Lezar <elezar@nvidia.com>
Compatibility review (advisory)New or updated breaking changes; see the full report for all findings. 1. cli — No explicit risk acceptance found
Review details
Advisory review; publication gates are unchanged. |
|
Compatibility risk acceptance: The changes to |
Passing the runner's 120 s limit to the program would stop a test before testkit's own step timeouts (up to 10 minutes) can report which step stalled, contrary to the program's documented contract. Keep program rows on the program's default until testkit learns the per-test deadline (#5333). Write nextest's grace period into the generated profile from the same constants, rather than relying on nextest's default. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
matthewgrossman
left a comment
There was a problem hiding this comment.
Approving the slimmed-down version. I pushed accad2d to this branch, and here is why.
What changed. As first written, this PR passed the runner's 120 s limit to the conformance program, while testkit's own step timeouts are longer (sandbox ready 10 min, CLI 5 min, stop 5 min, policy_advisor readiness 4 min). If this merged without #5455, a slow step would be killed with a generic timeout instead of failing with testkit's error, and the program's --timeout help ("Longer than testkit's own step timeouts, so those report first") would no longer hold. To make this PR safe to merge on its own, accad2d:
- drops
--timeout/--gracefrom the program command, so program rows keep the 900 s default for now; - keeps the nextest profile generated from the shared constants, and writes
grace-period = "10s"explicitly instead of relying on nextest's default; - removes
TEST_TIMEOUT_SECSandTEST_GRACE_SECSfor now, since nothing in this PR uses them; - updates the
RUN_TIMEOUTcomment and the test.
I've updated the title and description to match.
Plan for #5455. Once this lands, GitHub retargets #5455 to main. We'll merge main into its branch with a regular merge commit (no force push) and move the program limit there, alongside the deadline it adds:
- add
pub const TEST_TIMEOUT_SECS: u64 = SLOW_PERIOD_SECS * SLOW_TERMINATE_AFTER;back tobundle.rs, and make the grace constant shared; - pass
--timeout/--graceto the program inexecute.rs, next toOPENSHELL_TEST_TIMEOUT_SECS; - move the
assert_eq!(TEST_TIMEOUT_SECS, 120)check there.
That way program rows get the 120 s limit in the same change that makes testkit's waits fit inside it. #5458 then needs main (or #5455) merged in the same way.
Summary
The runner wrote its nextest profile as a hardcoded string. Generate it from named constants (a 30 s slow period, terminated after 4 periods, with a 10 s grace period) so #5455 can share the per-test limit with testkit and the program rows. The grace period is now explicit instead of relying on nextest's default.
Program rows keep the program's 900 s default here. Passing them the runner's 120 s limit would stop a test before testkit's own step timeouts (up to 10 minutes) can report which step stalled. #5455 caps those timeouts to the time a test has left, so it passes
--timeoutand--graceto the program in the same change.Related Issue
Part of #5416. The first item (an explicit per-test timeout for program rows) stays open until #5455 lands. Groundwork for #5333.
Changes
SLOW_PERIOD_SECS,SLOW_TERMINATE_AFTERandGRACE_PERIOD_SECS, and writegrace-periodexplicitly.slow-timeoutline.RUN_TIMEOUTcomment: program rows are bounded by the program's ownrun --timeoutdefault.Testing
cargo fmt --check,cargo clippy --all-targets -- -D warningsandcargo testpass fortests/runner.grace-period.