Skip to content

ci: compare release performance with a release PyPI serves - #1314

Merged
kvlonge merged 2 commits into
mainfrom
fix/release-performance-published-baseline
Nov 6, 2026
Merged

kvlonge merged 2 commits into
mainfrom
fix/release-performance-published-baseline

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Why

Release Performance fails on every push to main while the current version is tagged but not on PyPI. The job forced that version as the baseline, waited 30 minutes for it to publish, and then exited with a comparison error without measuring anything. The same applied to a release that was tagged but not yet published.

Changes

  • The baseline is the newest release PyPI actually serves. A main commit carrying an already released version uses that release once it is published, and the newest earlier release until then.
  • A release commit compares with the newest published release before it, never with itself.
  • The workflow passes --same-version-baseline for main commits that keep the version instead of forcing the version as the baseline.
  • A missing baseline or a failed install still fails the job.
  • No DuckShop case: this changes CI tooling, not compiler behaviour.

Verification

  • Unit tests for baseline selection: unpublished own release, published own release, release commit after publish, yanked release, first release.
  • Against live PyPI, a 0.187.3 candidate now resolves to 0.187.2.
  • make check-ci; the workflow step passes bash -n.
  • This PR's Release Performance run measures against 0.187.2.

Main commits carrying a version that is tagged but not yet on PyPI made
the job wait for that release and then fail without comparing anything.
The baseline is now the newest release PyPI actually serves: a main commit
uses its own release once published and the newest earlier one until then,
and a release commit never compares with itself.
@kvlonge
kvlonge enabled auto-merge (squash) November 5, 2026 23:40
@kvlonge
kvlonge merged commit 7a2c531 into main Nov 6, 2026
31 checks passed
@kvlonge
kvlonge deleted the fix/release-performance-published-baseline branch November 6, 2026 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant