Repository navigation
fix(ci): the pin check compared a digest that cannot be stable, so it never passed - #94
Merged
Merged
Conversation
… never passed #88 compared each reusable-workflow image pin against the digest the build had just published. It failed on 4 of 4 runs after it landed, including the 2026-10-26 SCHEDULED rebuild where nothing in the repo had changed and all six image jobs passed. The premise was mine and it was wrong: a rebuild with unchanged inputs does not produce a byte-identical digest. The Dockerfiles apt-get install current packages, so a rebuild days apart publishes a new digest. The check could never stay green past the next rebuild, and because the weekly cron rebuilds on its own, "pins behind" stopped carrying information. That is the cry-wolf signal this repo refuses elsewhere. So it now compares something that only moves when a human moves it. build-toolbox.yml stamps every image it publishes with a `git ls-files -s` fingerprint of that image's build inputs: toolbox/base, the image's own context, and toolbox/tools.lock. The check reads that label back off the PINNED image and compares it with the repo's inputs now. Four small registry calls per image, resolving the attestation-bearing index to its one real platform manifest and reading only the config blob, so no layers are pulled. Verified against the live registry before shipping: the chain resolves for all five images, and the shell fingerprint in the workflow and the Python one in the script agree on all five. More precise, too. #93 changed only toolbox/skill-audit/rules/, which under the old check turned all five pins red; under this one only skill-audit.yml is affected. A pinned image with no stamp is reported UNVERIFIABLE rather than passing, so this cannot report success on absent input, which is the failure mode it replaces. Moved out of build-toolbox.yml into its own workflow. A red build should mean the images are broken; the pins being behind is a different fact and should read as one, the same separation tool-drift.yml already uses. Not run on pull_request on purpose: the digest to pin does not exist until the publish has happened, so a PR that bumps a tool could never be green, and a check that cannot pass on the PR that causes it is one people learn to ignore. Also corrects the reproducibility claim in both cosign sign steps. Tolerating a Rekor 409 is still right, but a rebuild is byte-identical only when it hits the buildx cache, not whenever its inputs are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The pin check I added in #88 has failed on every run it ever had.
On the scheduled run, all six image jobs passed and only the pin job failed, with no human change involved.
What was wrong
My premise: a rebuild whose inputs have not changed produces a byte-identical digest. It does not. The Dockerfiles run
apt-get update && apt-get install, so a rebuild days apart pulls different packages and publishes a different digest. The check could never stay green past the next rebuild, and since the weekly cron rebuilds on its own, it was structurally always-red. "Pins behind" stopped carrying information, which is the cry-wolf signal this repo refuses elsewhere.What it compares now
Something that only moves when a human moves it.
build-toolbox.ymlstamps every image it publishes with agit ls-files -sfingerprint of that image's build inputs:security-toolbox-basetoolbox/base+toolbox/tools.locktoolbox/base+ its own context +toolbox/tools.locktoolbox/baseis in every domain stamp because each domain image is builtFROMthe base by digest.scripts/check-image-pins.pyreads that label back off the pinned image and compares it with the repo's inputs now. It resolves the attestation-bearing index to its one real platform manifest and reads only the config blob, so four small registry calls per image and no layers pulled.Verified before shipping
linux/amd64manifest plus oneunknown/unknownattestation manifest, and the script requires exactly one real platform rather than taking the first entry, because picking the wrong member is how two of five pins went in wrong in chore(workflows): move the five image pins alongside the tool bumps #82.toolbox/skill-audit/rules/, which under the old check turned all five pins red; under this one onlyskill-audit.ymlis affected.Expect this to be red on merge, correctly
Every currently pinned image predates the stamp, so all five report
UNVERIFIABLE:That is the honest state, not a defect. The pinned images also predate #91's tool bumps, so callers really are scanning with the previous toolchain. A pinned image with no stamp is reported
UNVERIFIABLErather than passing, because a check that reports success on absent input is the failure mode this replaces.The follow-up is the usual second half: once the build from this merge publishes stamped images, move the five pins to those digests. Each build job now prints the digest to pin in its step summary, which is what that follow-up needs.
Also
build-toolbox.ymlinto its ownImage pinsworkflow. A red build should mean the images are broken; the pins being behind is a different fact and now reads as one, the same separationtool-drift.ymlalready uses.Build toolbox imagesshould go green again on merge.pull_request, on purpose. The digest to pin does not exist until the publish has happened, so a PR that bumps a tool could never be green here, and a check that cannot pass on the PR that causes it is one people learn to ignore.