Repository navigation
fix(download): stop corrupting BuildKit layers when a cache download resumes - #316
Merged
Merged
Conversation
The Azure SDK, and with it BuildKit's type=gha cache, resumes a dropped download with only `x-ms-range: bytes=N-` and takes any 200 body as the bytes from offset N. The server only read `Range`, so a resume got the whole blob from byte 0 and BuildKit spliced it in at N. `x-ms-range` now wins over `Range`, like Azure Get Blob. A ranged read of an unmerged entry is served as a 206 from its Parts: Parts wholly before the range cost one ranged read answered 416, and the stream stops at the end of the range. Such a read no longer starts a Merge; the next whole read does. Closes #315 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…isting Skipping Parts with a 416 probe each cost one storage round trip per skipped Part, and BuildKit reopens a download on every read once it has resumed. `list_folder` now returns sizes by name, so the Part sizes from the existing listing locate the start Part directly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dKit resumes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
🚀 Deploying Preview to Cloudflare 🚀Preview URL: https://fix-download-x-ms-range-docs-gha-cache-server.louis-haftmann.workers.dev (commit 40cdd88)This URL reflects your latest Preview deploymentPreview Deployments by commit
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 Summary
BuildKit builds that used
cache-from: type=ghasometimes failed with corrupted layers (#315). BuildKit reads the cache through the Azure blob SDK. When a download drops, BuildKit retries it, and the SDK asks for the rest of the blob withx-ms-range: bytes=N-only. The server read onlyRange, so it answered with a 200 and the whole blob from byte 0. The SDK accepts any 200, and BuildKit wrote those bytes at offset N.Now
x-ms-rangewins overRange, as in Azure Get Blob. A ranged read never gets the whole entry with a 200:GET /download/{id} - range = Range + range = x-ms-range ?? Range merged or single Part -> 206 from the object (unchanged) - unmerged, several Parts -> 200 + whole entry, range ignored + unmerged, several Parts -> 206: one LIST gives the Part sizes, the read starts + in the Part where the range begins, then stops at its endA ranged read of an unmerged Storage Location no longer starts a Merge. The next full download still starts one, and BuildKit's first read of a blob is always a full download.
✅ Verification
Real BuildKit v0.34.1, end to end. A fresh
buildkitdexports the build cache to the server (filesystem storage, SQLite). A second freshbuildkitdimports it. A proxy cuts one 30 MB layer download after 5 MB.x-ms-rangeand gets 200 with the whole blob each time. The build fails:EAGER_MERGE=true(24 × 206). No ranged read got a 200.Tests, written first and failing on
dev:cargo +nightly fmt --checkandcargo clippy --all-targets --locked: clean.cargo test: 83/83 on SQLite with filesystem, S3, GCS and Azure Blob storage.pnpm check && pnpm buildinwebsite/: passes.Benchmark (
cargo run --release --example benchmark), with images built fromdevand from this branch:First download changes the most between runs. Over 3 runs it measured 6840–7336 MiB/s on
devand 6407–7679 MiB/s on this branch. The ranges overlap, so the difference is noise.🔍 Reviews
list_foldernow returns sizes by name, so the LIST the read already makes gives the start Part.🔗 Issues
Closes #315
💥 Merge danger
Door: two-way. No schema or stored data changes.
Blast Radius: downloads
Every download goes through this path. Full downloads keep their old code path. Only ranged reads of unmerged multi-Part entries change their answer, from 200 to 206.
ENABLE_DIRECT_DOWNLOADS, S3 and GCS signed URLs ignorex-ms-range: the RustFS and fake-gcs test services answer 200 with the whole object. The server cannot fix this, soconfiguration.mdnow tells BuildKit users on S3 or GCS to leave direct downloads off. Azure Blob is not affected.go-actions-cache,readerAtCloserdoes not set its offset after it reopens a download. So after one resume, every later read reopens the download. That is an upstream bug in tonistiigi/go-actions-cache.🤖 Generated with Claude Code