Skip to content

feat(ingress): record source metadata in the image - #121

Open
kvinwang wants to merge 6 commits into
mainfrom
feat/ingress-image-metadata
Open

feat(ingress): record source metadata in the image#121
kvinwang wants to merge 6 commits into
mainfrom
feat/ingress-image-metadata

Conversation

@kvinwang

@kvinwang kvinwang commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

A published dstacktee/dstack-ingress@sha256:… currently carries nothing that points back to the code that built it (config Labels: null, no manifest annotations; /etc/GIT_REV existed but nothing read or documented it). This PR makes the image self-describing while keeping the digest reproducible between CI and a local ./build-image.sh.

The same key/value set is recorded in three places:

Location How to read it
Image config labels skopeo inspect docker://<ref> | jq .Labels, docker inspect
Image manifest annotations skopeo inspect --raw docker://<ref> | jq .annotations
/etc/dstack-ingress/build-info (replaces /etc/GIT_REV) cat inside the container; also printed as the first line of the container log

Keys are the standard OCI ones: source, revision, version, url, documentation, title, description, licenses, base.name, base.digest.

Reproducibility

Every value is derived from the git checkout only: git rev-parse HEAD, the new committed VERSION file, the Dockerfile FROM digest, and a hard-coded canonical SOURCE_URL (overridable for forks). No build time, CI run id or runner identity goes in, so the digest of a clean checkout is stable.

  • build-image.sh warns and marks revision as <sha>-dirty when the tree has uncommitted or untracked changes. CI passes --require-clean.
  • VERSION is a committed file rather than git describe, so git checkout <sha> && ./build-image.sh reproduces the digest even without tags fetched. Bumping the version is now a source change before tagging (set to 2.5 for the next release).

Bonus: fixes a silent digest divergence

scripts/__pycache__ is gitignored, and the Dockerfile copies scripts/ wholesale into /scripts. So any developer who had run the test suite baked stray .pyc files into their image — with no dirty-tree warning and a digest quietly different from CI's. This is the same class of bug as the chmod 0644 requirements.txt caveat in the v2.4 release notes. The new .dockerignore fixes it (and keeps the 120 MB oci.tar out of the build context).

Release workflow

The version gate now derives the version from VERSION and requires the ref to be a matching dstack-ingress-v* tag. Previously a workflow_dispatch run parsed the version as the literal string refs/heads/main and only failed much later at the registry push. The release body gained the source commit, how to read the metadata, and the reproduce commands.

Note: only manifest: annotations are used. BuildKit v0.20.2 rejects index:/index-descriptor: annotations for single-platform OCI export, and manifest-descriptor: only lands in the local index.json, which skopeo copy does not push.

Testing

Determinism and the .dockerignore fix

Two full builds at a66b950, the second with a simulated developer tree (gitignored scripts/__pycache__/*.pyc, scripts/tests/__pycache__/*.pyc, .pytest_cache/ present):

Build Tree Digest
1 clean sha256:bb0a3aa8f5dbd1ea9221770b7c82723ffda67546c575025fb73e2b247f03fc24
2 dev artifacts present sha256:bb0a3aa8f5dbd1ea9221770b7c82723ffda67546c575025fb73e2b247f03fc24

Identical, which covers both determinism and the .dockerignore fix. Reproduce on a native linux/amd64 host:

git checkout a66b950
cd custom-domain/dstack-ingress && ./build-image.sh
skopeo inspect oci-archive:./oci.tar | jq -r '.Digest'

Metadata survives the push to a registry

Docker v2s2 manifests have no annotations field, so this had to be verified rather than assumed. Pushed the built image through a local registry with the exact command CI uses (skopeo copy --insecure-policy oci-archive:./oci.tar docker://…):

before push in registry
digest sha256:d4ce223a… sha256:d4ce223a… (unchanged)
mediaType application/vnd.oci.image.manifest.v1+json same
manifest annotations 10 10
config labels 10 10

The digest being unchanged also matters for the existing attestation step, which takes subject-digest from the local oci.tar. Cross-checked against the real world: the published dstacktee/dstack-ingress:2.4 manifest is OCI format and its registry digest matches the v2.4 release notes exactly, so skopeo already preserves the format today.

Other checks

  • manifest annotations byte-identical to the config labels; revision equals HEAD, version is 2.5
  • no __pycache__ under /scripts in the resulting image
  • /etc/dstack-ingress/build-info matches; container log starts with dstack-ingress 2.5 revision … source https://github.com/Dstack-TEE/dstack-examples
  • --require-clean with an untracked file under scripts/ fails before building and leaves no .BUILD_INFO behind
  • working tree still clean after both builds (pinned-packages.txt unchanged)
  • version gate simulated for dstack-ingress-v2.5 (pass), dstack-ingress-v9.9 (fail, mismatch), refs/heads/main (fail, not a tag)
  • --annotation needs Buildx ≥ 0.12; ubuntu-24.04 runners ship 0.36.1
  • no test parses the container's stdout positionally, so the new first log line breaks nothing
  • release body parsed out of the workflow YAML and checked for correct fencing and indentation

Follow-up

docker-compose.yaml / docker-compose.multi.yaml still pin 2.3; they get bumped to the real digest after the release build, as in dc34a5d.

Add OCI labels, manifest annotations and /etc/dstack-ingress/build-info
pointing back to the source repository, commit, version and pinned base
image, so a published digest can be traced to the code that built it.

All values derive from the git checkout (commit, the new VERSION file,
the Dockerfile FROM line), so rebuilding a clean checkout of the same
commit still yields the same digest. The build script warns and marks
the revision -dirty when the tree has uncommitted or untracked changes;
CI passes --require-clean and checks that the release tag matches
VERSION. The entrypoint prints the build info as its first log line.

Also add a .dockerignore so untracked local artifacts (oci.tar,
__pycache__, .pytest_cache) neither inflate the build context nor leak
into the image.
Copilot AI lite review requested due to automatic review settings September 5, 2026 10:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Only minor documentation/comment wording adjustments are suggested; the functional changes appear consistent and complete.

Pull request overview

This PR makes the custom-domain/dstack-ingress image self-describing by embedding standard OCI source metadata into (1) image config labels, (2) manifest annotations, and (3) an in-image /etc/dstack-ingress/build-info file, while preserving reproducible digests between CI and local builds.

Changes:

  • Add a committed VERSION file and enforce tag ↔ VERSION consistency in the release workflow.
  • Extend build-image.sh to generate consistent OCI labels/manifest annotations and emit the same key/value set into .BUILD_INFO copied into the image.
  • Print the build identification line on container startup and document how to inspect the metadata.
File summaries
File Description
custom-domain/dstack-ingress/VERSION Introduces committed version source for reproducible builds and release tagging.
custom-domain/dstack-ingress/scripts/entrypoint.sh Prints build identification from /etc/dstack-ingress/build-info early at startup.
custom-domain/dstack-ingress/README.md Documents OCI metadata locations/keys and reproducible build workflow.
custom-domain/dstack-ingress/Dockerfile Copies generated .BUILD_INFO into the image as /etc/dstack-ingress/build-info.
custom-domain/dstack-ingress/build-image.sh Generates metadata, enforces clean-tree behavior (optional), and applies labels/annotations deterministically.
custom-domain/dstack-ingress/.gitignore Ignores the generated .BUILD_INFO artifact.
custom-domain/dstack-ingress/.dockerignore Prevents large/local artifacts from entering build context and impacting digest.
.github/workflows/dstack-ingress-release.yml Enforces VERSION matching and builds releases with --require-clean, plus improved release notes.
Review details
  • Files reviewed: 8/8 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread custom-domain/dstack-ingress/.dockerignore Outdated
Comment thread custom-domain/dstack-ingress/build-image.sh Outdated
kvinwang and others added 5 commits September 5, 2026 18:50
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
The version gate parsed the version out of GITHUB_REF unconditionally,
so a workflow_dispatch run took the literal string 'refs/heads/main' as
the version and only failed later at the registry push. Derive the
version from the committed VERSION file, which is what the image
records, and require the ref to be a dstack-ingress-v* tag that matches
it.
The reproduce instructions dropped the prerequisite line the v2.4 notes
carried. The pinned toolchain only reproduces the published digest on a
native linux/amd64 host, so someone following the steps on arm64 would
get a mismatch with nothing explaining why.
dstack-ingress-v2.5 was tagged and published on 2026-09-03, so 2.5 is
taken: docker.io/dstacktee/dstack-ingress:2.5 already exists at
sha256:97285855. Releasing 2.5 again would either fail on the existing
tag or overwrite a published digest that users may have pinned.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants