Skip to content

Add Docker-style image tagging - #425

Open
chruffins wants to merge 13 commits into
mainfrom
hypeship/image-tag
Open

Add Docker-style image tagging#425
chruffins wants to merge 13 commits into
mainfrom
hypeship/image-tag

Conversation

@chruffins

@chruffins chruffins commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

summary

Kernel runs browser sessions and app deployments as VM-backed instances. This PR adds Docker-style local image tagging and separates immutable bootable image content from mutable repository/tag references.

The result is one local rootfs per content digest, with repository tags acting as lightweight references. A ready Hypeman image can be retagged and pushed without pulling or reconverting it, while existing legacy images remain readable during the layout transition.

1. pre-existing flows + new flows

pre-existing image pull flow

createImage(name):
  ref = resolveRegistryReference(name)
  digest = ref.manifestDigest

  if metadata[digest] exists:
    if status == failed:
      remove failed image state
    else:
      return existing image

  write metadata:
    status = pending
    name = ref
    digest = digest

  enqueue build:
    pull OCI layers
    convert rootfs to erofs/ext4
    write rootfs disk
    update metadata to ready
    create tag -> digest symlink

pre-existing lookup flow

getImage(name):
  ref = parse(name)

  if ref is a tag:
    digest = resolve tag symlink
  else:
    digest = ref.digest

  read metadata for digest
  verify ready images have a rootfs disk
  return image metadata

pre-existing delete flow

deleteImage(name):
  resolve name to digest

  if name is a tag:
    remove tag symlink
    if no tags reference digest:
      remove digest directory
  else:
    remove all repository tags for digest
    remove digest content if no references remain

new local tagging flow

tagImage(source, target):
  sourceRef = parse(source)
  targetRef = parse(target)

  require targetRef includes a tag

  if source is a tag:
    digest = resolve source tag
  else:
    digest = source digest

  metadata = read source digest
  require metadata.status == ready

  if source repository == target repository:
    create/update target tag symlink
  else:
    ensure digest exists in shared content layout
    promote legacy content if necessary
    move source tags to shared content references
    create target repository tag reference

  return image with target name

This allows:

hypeman tag alpine:latest registry.example.com/app:v1
hypeman push registry.example.com/app:v1

The operation does not pull or reconvert the image.

2. changes to the data model

There is no database schema or migration. The main change is separating image content from image references.

API model

Added:

POST /images/{name}/tag

request:
  {
    "target": "registry.example.com/app:v1"
  }

response:
  image

The endpoint returns explicit errors for invalid references, missing images, and non-ready sources.

internal model

The existing image metadata fields remain compatible:

metadata:
  name
  digest
  platform
  status
  size
  image configuration
  build state
  timestamps

The ownership model changes conceptually:

content[digest]   -> one canonical rootfs and metadata record
repository/tag   -> reference to content[digest]

Tags are still represented by filesystem references rather than a new database table. An image can therefore have multiple repository/tag references without duplicating its metadata or rootfs.

This matches the app platform’s existing deployment model: deployments persist both an image reference and an image digest, while app versions identify the deployable version. The reference remains a convenient name; the digest identifies the immutable image used by the VM.

3. changes to filesystem layout

legacy layout

images/
└── <repository>/
    ├── <tag> -> <digest>
    └── <digest>/
        ├── metadata.json
        └── rootfs.erofs

On macOS, the rootfs uses rootfs.ext4.

The legacy layout stores a separate digest directory under each repository.

content-addressed layout

images/
├── content/
│   └── <digest>/
│       ├── metadata.json
│       └── rootfs.erofs
└── repositories/
    └── <repository>/
        └── <tag> -> ../../content/<digest>

The content directory is shared across repositories. Repository tags are references into that shared content directory.

compatibility behavior

Readers understand both layouts:

read:
  use a ready legacy image if content state is incomplete
  otherwise use ready shared content
  fall back to legacy content when no shared content exists

New writes use the content-addressed layout. Existing legacy images are promoted when a cross-repository tag requires shared ownership.

Deletion distinguishes between:

remove a tag reference
remove a repository-local digest
remove shared content only when no tags or digest references remain

4. justifications for the filesystem change

keep the bootable browser image immutable

Hypeman does not run the OCI layer graph directly. It converts the image into a bootable EROFS or EXT4 rootfs disk, which is consumed by the VM that runs a Kernel browser session.

That disk is the expensive, host-local runtime artifact. Repository tags such as chrome:stable, customer-specific names, or deployment names are mutable references to it. They should not create another rootfs or change the bytes used to start the browser.

one converted rootfs -> many names

make app deployment identity stable

The app platform records both image_ref and image_digest for a deployment and uses the image to create a deploy-mode instance. The reference is useful for display and lookup, but the digest is the stable identity needed to reproduce, roll out, or roll back an app version.

Content-addressed storage keeps that identity local to the digest. Moving a tag or adding a deployment-specific alias does not create a second image or change which bytes an app version refers to.

support host-local caching for browsers and apps

A browser session or app invocation may start on a different host from the one that first converted the image. Each host needs a local cache of the same immutable bootable content.

With the shared layout, a host can:

receive digest
check images/content/<digest>
materialize a repository/tag reference if needed
start the VM from the cached rootfs

The host does not need to preserve every repository name that previously referenced the image.

avoid duplicate rootfs data

A single digest can be referenced by multiple repositories, app versions, and browser-facing names. Keeping content under the repository path creates duplicate rootfs files as aliases are added.

The shared layout gives each digest one local content owner while allowing any number of references. Hard-linked aliases also let disk accounting count the physical rootfs once.

make deletion and rollback safe

Deleting a tag must not remove content still needed by another app version, browser session, or digest-only deployment. Conversely, an old app version should be able to roll back to its digest even after a mutable tag moves forward.

Separating content from references lets cleanup answer the correct question:

are there any remaining references to this digest?

rather than inferring ownership from one repository directory.

enable atomic promotion and compatibility

Legacy content can be promoted into the shared layout by:

hard-link rootfs into content/<digest>
write shared metadata
repoint repository tags
remove the legacy digest directory

The rootfs is not copied, and incomplete content does not become authoritative over a ready legacy image. Dual-layout readers allow existing browser and app images to continue working while new writes use the shared layout.

enable future lifecycle operations

The shared content boundary gives later browser and app platform operations a clear object to manage:

  • prewarming a digest on selected hosts
  • evicting unused content without removing active references
  • verifying or repairing one rootfs per digest
  • replicating content between hosts
  • backing up and restoring immutable app/browser images
  • scanning a converted image once per digest

One follow-up constraint remains: if the same OCI digest can produce different rootfs bytes for different platforms, output formats, or converter versions, the eventual content key should include those dimensions rather than using the OCI digest alone.

Comment thread openapi.yaml
Comment thread lib/images/storage.go
@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown
-->

✱ stlc build

go code · compare

Your SDK build was successful.

generate ✅bootstrap ✅format ✅

116 files generated at e7a8295 (pushed)

go get github.com/kernel/hypeman-go-staging@e7a8295c10542dc81efd2ead2df89917585d8087
python code · compare

Your SDK build was successful.

generate ✅bootstrap ✅format ✅

231 files generated at ef2ae23 (pushed)

typescript code · compare

Your SDK build was successful.

generate ✅bootstrap ✅format ✅

138 files generated at de1c2a4 (pushed)

Diagnostics: ❗ 0 new / 1 total error, 💡 0 new / 5 total note
LevelCodeMessageTargets
Build metadata
Buildbd_769vMc2j-strong-fringe
Timestamp2026-08-19T20:09:40.582Z
stlc8413509
Spec hashc0136e7a9a1e
Config hash659c3687c3f0

This comment is auto-generated by stlc and is kept up to date as you push.
If you push new commits, re-run this workflow to update this comment.
Last updated: 2026-08-19 20:10:14 UTC

Comment thread lib/images/disk_usage.go
@chruffins
chruffins marked this pull request as ready for review August 19, 2026 15:26
@chruffins
chruffins requested a review from sjmiller609 August 19, 2026 15:26
@chruffins

Copy link
Copy Markdown
Contributor Author

companion PR here: kernel/hypeman-cli#65

@sjmiller609

Copy link
Copy Markdown
Collaborator

mentioned to assess this directory layout:

images/
├── docker.io/library/alpine/       # existing legacy layout, untouched
│   ├── latest -> <digest>
│   └── <digest>/
│       ├── metadata.json
│       └── rootfs.erofs
│
├── content/                        # new layout only
│   └── <digest>/
│       ├── metadata.json
│       └── rootfs.erofs
│
└── repositories/                   # new tags only
    └── example.com/app/
        └── v1 -> ../../../content/<digest>

and if the migration is worth it or if we should stick to existing for now. un-requesting review until pinged again.

@sjmiller609
sjmiller609 removed their request for review August 19, 2026 18:42
Comment thread lib/images/storage.go

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a170537. Configure here.

Comment thread lib/images/storage.go
Comment thread lib/images/storage.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants