Skip to content

Add elfuse-oci pull into a local image store - #341

Open
henrybear327 wants to merge 2 commits into
sysprog21:mainfrom
henrybear327:oci/pull
Open

Add elfuse-oci pull into a local image store#341
henrybear327 wants to merge 2 commits into
sysprog21:mainfrom
henrybear327:oci/pull

Conversation

@henrybear327

@henrybear327 henrybear327 commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Summary by cubic

Adds elfuse-oci, a Go CLI that pulls OCI images into a local store, keeping elfuse an OCI-agnostic runtime.

New Features

  • pull fetches an image via go-containerregistry into an OCI layout and pins it in index.json by canonical ref and platform; a legacy refs.json store is refused.
  • A single flock serializes whole pulls, and the pin table is rewritten via a synced temp file and rename, so a crash can't publish a truncated table.
  • --timeout bounds the whole pull including the lock wait; a lock freed after the deadline is released, not used. --platform only accepts linux/arm64 and linux/amd64.
  • A bare manifest or index child that states no platform is pinned only when its config's platform matches; a child that is itself an index is refused.
  • Credentials come from Docker config inline auths or helpers, with the pull anonymous when no entry matches.
  • mk/oci.mk opts the CLI into all, check, and lint only when PATH's Go meets go.mod under GOTOOLCHAIN=local; CI runs race tests and lint on Linux.
  • Documents the store, pull flow, and validation with README and dedicated testing targets.

Written for commit b96a08d. Summary will update on new commits.

Review in cubic

@henrybear327
henrybear327 requested a review from jserv August 29, 2026 01:12
@henrybear327 henrybear327 self-assigned this Aug 29, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 20 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread cmd/elfuse-oci/creds.go Outdated
Comment thread cmd/elfuse-oci/creds.go Outdated
Comment thread go.mod Outdated
Comment thread cmd/elfuse-oci/pull.go Outdated
Comment thread cmd/elfuse-oci/flock.go
Comment thread mk/oci.mk Outdated
Comment thread mk/oci.mk Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 20 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread cmd/elfuse-oci/store.go Outdated
Comment thread cmd/elfuse-oci/pull.go Outdated
Comment thread cmd/elfuse-oci/creds.go Outdated
Comment thread mk/oci.mk
Comment thread cmd/elfuse-oci/pull.go Outdated
Comment thread cmd/elfuse-oci/helpers_test.go

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 20 files (changes from recent commits).

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.
Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread cmd/elfuse-oci/pull_test.go Outdated
Comment thread cmd/elfuse-oci/main_test.go Outdated
Comment thread cmd/elfuse-oci/creds.go Outdated
Comment thread cmd/elfuse-oci/store.go Outdated
Comment thread mk/oci.mk Outdated
Comment thread .github/workflows/build.yml

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 20 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name=".github/workflows/build.yml">

<violation number="1" location=".github/workflows/build.yml:135">
P2: On a cold runner this job compiles the containerd/v2+grpc+protobuf dependency tree three separate times with an empty build cache: `make elfuse-oci` (non-race), oci-lint's `GOOS=darwin GOARCH=arm64 go vet` (a separate cache key), and oci-test's `go test -race` (a near-full recompile under race instrumentation). Ten minutes may be too tight and can turn into intermittent job timeouts that require re-runs. Raise the budget or persist the Go build/module cache (e.g. actions/cache on GOMODCACHE/go-build) as the other jobs do for Homebrew downloads.</violation>
</file>

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread cmd/elfuse-oci/flock.go
Comment thread cmd/elfuse-oci/common.go
oci-linux:
name: elfuse-oci (Linux)
runs-on: ubuntu-24.04
timeout-minutes: 10

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: On a cold runner this job compiles the containerd/v2+grpc+protobuf dependency tree three separate times with an empty build cache: make elfuse-oci (non-race), oci-lint's GOOS=darwin GOARCH=arm64 go vet (a separate cache key), and oci-test's go test -race (a near-full recompile under race instrumentation). Ten minutes may be too tight and can turn into intermittent job timeouts that require re-runs. Raise the budget or persist the Go build/module cache (e.g. actions/cache on GOMODCACHE/go-build) as the other jobs do for Homebrew downloads.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/build.yml, line 135:

<comment>On a cold runner this job compiles the containerd/v2+grpc+protobuf dependency tree three separate times with an empty build cache: `make elfuse-oci` (non-race), oci-lint's `GOOS=darwin GOARCH=arm64 go vet` (a separate cache key), and oci-test's `go test -race` (a near-full recompile under race instrumentation). Ten minutes may be too tight and can turn into intermittent job timeouts that require re-runs. Raise the budget or persist the Go build/module cache (e.g. actions/cache on GOMODCACHE/go-build) as the other jobs do for Homebrew downloads.</comment>

<file context>
@@ -126,6 +126,30 @@ jobs:
+  oci-linux:
+    name: elfuse-oci (Linux)
+    runs-on: ubuntu-24.04
+    timeout-minutes: 10
+    steps:
+      - name: Checkout
</file context>
Suggested change
timeout-minutes: 10
timeout-minutes: 20

Comment thread cmd/elfuse-oci/helpers_test.go

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 20 files (changes from recent commits).

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.
Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread cmd/elfuse-oci/main.go Outdated
Comment thread cmd/elfuse-oci/creds.go Outdated
Comment thread cmd/elfuse-oci/creds.go Outdated
Comment thread .gitignore Outdated
@henrybear327
henrybear327 marked this pull request as draft August 29, 2026 22:50
@henrybear327
henrybear327 marked this pull request as ready for review August 29, 2026 23:44

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 22 files

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

Re-trigger cubic

Comment thread cmd/elfuse-oci/pull.go Outdated
Comment thread cmd/elfuse-oci/pull.go Outdated
Comment thread cmd/elfuse-oci/store.go Outdated
Comment thread cmd/elfuse-oci/store_test.go
Comment thread cmd/elfuse-oci/main.go Outdated
@henrybear327

Copy link
Copy Markdown
Collaborator Author

Final switch of the open source library after trying out different projects and attempting to make it maintainable in the long run

Publish canonical references and per-platform manifests through standard
OCI indexes so layout readers can resolve the stored image. Use
go-containerregistry for reference parsing and registry access.
Describe the OCI store, reference normalization, platform selection,
credentials, timeouts, and pull validation. Add the command and
environment surface to the user guide.
@henrybear327
henrybear327 force-pushed the oci/pull branch 2 times, most recently from a5146cf to b96a08d Compare August 30, 2026 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant