Repository navigation
test(perf): scale the dense compile guards to the runner and judge warm and edit on medians - #1395
Merged
Merged
Conversation
…ee runs The warm and one-model edit budgets were checked against one compile each, so a single slow moment on a hosted runner failed changes that leave the compiler untouched. The guard now runs three warm compiles and three edits in the same job and compares their medians with the unchanged budgets. Every run still checks diagnostics, peak memory and cache behaviour; warm runs keep the cold fingerprint and the final edit still matches the from-scratch oracle.
kvlonge
enabled auto-merge (squash)
November 9, 2026 19:36
kvlonge
disabled auto-merge
November 9, 2026 19:38
Hosted runners differ by CPU class, and the same compiler measures about a third slower on the slower class, so the dense guard's absolute budgets judged the runner rather than the code. Before each dense cold and warm/edit guard, the job now times a fixed interpreter workload that does not depend on SQLBuild, in a fresh process, and scales the wall budgets by its fastest repetition over the reference time, floored at 1.0 so a faster runner keeps today's budgets. The calibration times, CPU model, factor and effective budgets are printed with the measurements. Same-runner ratio jobs remain the regression detector.
…amples Across EPYC 9V45, 9V74 and 7763 and Xeon 8573C and 8370C runners the calibration's fastest repetition ranged from 0.31 s to 0.67 s. The reference is the top of the classes the dense budgets already fit (0.55 s), so those keep factor 1.0 and only measurably slower runners get a larger budget.
kvlonge
changed the title
kvlonge
enabled auto-merge (squash)
November 9, 2026 21:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The dense compile guards compare wall time with absolute budgets. Hosted runners come in CPU classes whose speed differs by up to 1.6x, so the guards judged the runner rather than the compiler: unchanged code from main failed the 5,000-model budgets on slower runners.
Changes
Verification
make check-cipassed. Dense 5,000-model job on this branch, every run passing:EPYC 9V74 (0.46-0.52 s) and Xeon 8573C (0.54 s) were measured on the 1,000-, 3,000- and 10,000-model jobs and keep factor 1.0.
Against the 9V45, the factor over-reads the compile slowdown (7763: 1.86x against 1.49x warm; 8370C: 2.0x against 1.32x). That leaves slower runners slack and never undershoots.