Skip to content

ci: judge release peak memory on the worst of five runs - #888

Merged
kvlonge merged 3 commits into
mainfrom
ci/release-rss-worst-case
Sep 30, 2026
Merged

kvlonge merged 3 commits into
mainfrom
ci/release-rss-worst-case

Conversation

@kvlonge

@kvlonge kvlonge commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

Why

The release comparison failed the 0.126.6 release PR on compile --no-cache peak RSS (median 622 -> 830 MiB), but peak RSS is bimodal on both versions (~640 or ~860 MiB, depending on whether concurrent phases overlap). A 7-run local check showed the worst case did not grow (890 -> 868 MiB); the median of 3 was flipping between levels.

Changes

  • Wall and CPU stay medians; peak RSS is judged on the worst run.
  • Default runs per version 3 -> 5 (about 8 more minutes on release PRs, well inside the job timeout).
  • Report text and COMPILE_BENCHMARKS.md describe the rule; unit tests cover the bimodal case passing, a single high candidate run failing, and a larger worst run failing.

Verification

  • Focused release-performance unit and E2E tests; make check-ci; one independent review.
  • Residual risks (accepted, recorded on CHI-440): growth only in the low RSS level is hidden behind the high level; one high baseline run raises the bar. CHI-440 makes peak memory deterministic.

@kvlonge
kvlonge enabled auto-merge (squash) September 30, 2026 16:25
@kvlonge
kvlonge merged commit 7c904e9 into main Sep 30, 2026
29 checks passed
@kvlonge
kvlonge deleted the ci/release-rss-worst-case branch September 30, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant