Repository navigation
fix(mirror): paginate python-build-standalone release discovery - #277
Merged
Merged
Conversation
Each python-build-standalone release now lists ~900 assets (~2MB of JSON), so requesting 100 releases in one GitHub API call times out with HTTP 504. The sync logged this as a warning, found zero python jobs, and still exited successfully, so no new Python versions have been mirrored since December 2025. - Fetch releases in pages of 10, following the Link header, still bounded to the newest 100 releases - Authenticate GitHub API calls with GITHUB_TOKEN when set, to avoid the shared unauthenticated rate limit on CI runners - Keep mirroring jobs from healthy sources when one fails, but exit non-zero so a broken source fails the workflow instead of passing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Python 3.13.16 (and every other Python release since December 2025) is missing because the daily Mirror Sync has been silently discovering zero Python versions.
PythonStandaloneSourcerequestedreleases?per_page=100from the GitHub API in a single call. Each python-build-standalone release now carries ~900 assets (~2MB of JSON), so that response would be ~190MB and GitHub returns HTTP 504. The tool logged it as a warning, processed 0 jobs, and exited 0, so the workflow stayed green and the manifest generator kept running on stale data. The last 40 runs all showfailed to fetch from python-build-standalone: fetching releases: HTTP 504(one showsHTTP 403, a rate limit), andpython.jsonhasn't changed since Dec 16, 2025.per_pageChanges
Linkheader, still bounded to the newest 100 releases (same coverage as before)GITHUB_TOKEN(only toapi.github.com) when set;mirror-sync.ymlnow passes the workflow tokenhttptestserver, token scoping, and the new per-source error collectionVerification
go test -race ./...inscripts/mirror-binariespasses;./rnr checkpasses--runtime=python --dry-run) finishes in ~70s and discovers 543 jobs, includingpython/3.13.16/windows-amd64.tar.gzNotes
runtime: pythondelivers the missing versions without waiting for the 4 AM UTC schedule. Manifest generation follows automatically. The CLI reads manifests frommanifests.dtvem.io, so no CLI release is needed.Resolves #276