Skip to content

test(compile): tighten compile performance budgets to CI measurements - #1160

Closed
kvlonge wants to merge 1 commit into
mainfrom
test/tighten-compile-budgets
Closed

kvlonge wants to merge 1 commit into
mainfrom
test/tighten-compile-budgets

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Why

The compile performance budgets were loose enough to let a regression through. Each one now sits at about 1.2x the slowest of two CI runs (39243632283, 39251812250), and never below 1.15x the slowest recent CI run of the same code, so runner noise doesn't fail them.

Changes

Wall limits in seconds (old → new, slowest of the two runs in brackets):

Guard 3k 5k 10k
fresh cold 9.5→8 (6.5) 16→12.5 (10.2) 31→25.5 (20.9)
cache cold 10.5→9 (7.5) 18→14.5 (11.4) 34.5→28 (23.1)
cache warm 6→4.5 (3.5) 10.5→7 (5.6) 19.5→14 (11.3)
cache edit 6→4.5 (3.5) 10.5→7 (5.5) 19.5→13.5 (11.1)
dense cold 16.5→12 (9.9) 26→18 (13.1) 51→35 (29.1)
dense warm 8→6 (4.7) 13→9 (6.8) —
dense edit 8.5→6 (4.8) 13.5→9.5 (6.9) —
delta wall 4.5→2 (1.6) — 9→8 (6.3)
delta CPU ratio 0.375→0.30 (0.25) — 0.30→0.275 (0.21)
delta full timeout 15→7 (5.6) — 60→28 (23.1)
  • Dense cold 1k: 7.5→6 (4.7). Varied cache 1k: cold 24→9 (7.3), warm 10→2.5 (1.9), edit 13→2.5 (1.9).
  • Peak RSS: fresh 3k/5k 1.5/1.75→1.06/1.63 GiB (0.87/1.30); cache 3k/5k/10k 2/2/3→1.13/1.63/2.94 GiB (0.90/1.35/2.40); varied 2 GiB→832 MiB (665 MiB).
  • Limits already below the floor stay as they are: dense RSS, fresh 10k RSS. The 1.05 fallback CPU limit, the 4 GiB cgroup and the same-runner ratio limits are also unchanged.

Verification

  • The changed performance tests collect.
  • ruff, ty and fensu are clean, and the hygiene scan is clean.
  • CI's performance jobs check the new budgets on this pull request.

…CI measurements

Each absolute budget is now about 1.2x the slowest of two CI runs of this
branch, and never below 1.15x the slowest recent CI run on the same code (runs
based on 0.158.2 or later). That covers the fresh-process, cache, varied-cache,
dense cold, warm and edit, and delta edit-loop wall budgets, the delta CPU
ratios and the peak RSS limits. A limit already below that floor is
unchanged. The fallback CPU limit, the 4 GiB cgroup and the same-runner ratio
limits are unchanged.
@kvlonge
kvlonge enabled auto-merge (squash) October 19, 2026 14:21
@kvlonge

kvlonge commented Oct 19, 2026

Copy link
Copy Markdown
Contributor Author

Folded into #1158 so the bug fixes, the Polyglot set-operation fix and the tightened budgets ship in one PR.

@kvlonge kvlonge closed this Oct 19, 2026
auto-merge was automatically disabled October 19, 2026 14:29

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant