Skip to content

prepare-venue passes a deploy KEY to build:generate, so a qualified pod bed fails with unknown box "charly/check-sidecar-pod" #46

Closed

Description

@aitrawog

Problem

A disposable pod bed invoked QUALIFIED from the umbrella cannot deploy: prepare-venue asks build:generate resolve for the bed's slash-form deploy key as if it were a BOX name, so the deploy dies at unknown box "charly/check-sidecar-pod" before any container exists.

This is the same site as plugin-build#28 (sdk/deploykit/graph.go:312, FilterBox), but not the same string as the one #28 fixed.

Measured

charly check run charly.check-sidecar-pod from the umbrella root, 2026-11-03:

[vm-build-check-sidecar-pod-ephvm] PASS   3m45.933s
[image-build]                      PASS  10m59.609s
[check-image]                      PASS   2m0.379s
[deploy-add]                       FAIL  20m45.57s: exit 1
charly: error: command "deploy": deploy-dispatch add: deploy plugin: deploy-dispatch add:
  prepare venue: InvokeProvider deploy:pod op=prepare-venue:
  plugin-deploy-pod prepare-venue: build-generate-resolve:
  build generate resolve: scoping generation to requested boxes:
  unknown box "charly/check-sidecar-pod"

Why this differs from #28

#28 reported unknown box "openclaw" — a bare NAMESPACE, and it was fixed in opencharly/plugin-check#109 by converting the roster address once in bedAdd:

if len(args) > 0 && !isRemoteImageRef(args[0]) { args[0] = spec.DeployKey(...) }   // bed_run.go:197

That fix makes the argument a valid deploy key. This failure is the next consequence: charly/check-sidecar-pod is a deploy key, and the deploy's image is check-k8s-deploy-app (charly/charly.yml:1243 — check-sidecar-pod: pod: {image: check-k8s-deploy-app, disposable: true}). A deploy key is not a box name, so handing it to build:generate resolve scopes generation to a box that cannot exist.

RCA

  • Mechanism. prepare-venue resolves what to build from the deploy, and what reaches FilterBox is the deploy key rather than the deploy's image:. FilterBox (sdk/deploykit/graph.go:309-312) correctly rejects a name that is not in the box map — the wrong input, not a wrong check.
  • Missed control. A qualified bed address reaches FilterBox as its NAMESPACE: pod bed deploys fail at prepare-venue with unknown box "openclaw" #28's fix was verified against the string it was reported with (a bare namespace) and closed; nothing asserts the next string in the same chain (the deploy key) is a box name. The arm that would have caught it is "a qualified pod bed with an image: different from its own name reaches prepare-venue".
  • Blast radius. Every qualified pod bed whose deploy name differs from its image — including charly.check-sidecar-pod, which is the reproduction for opencharly/sdk#362 and therefore blocks that PR's A2 witness.
  • Root fix. At the prepare-venue call site, resolve the deploy's image: and pass THAT to build:generate resolve; a deploy key must never reach the box namespace. Regression arm: the bed above must reach [deploy-add] PASS.

Note

I have not established whether this is a regression from a later pin or a case #109 never covered — both are consistent with the evidence. Whoever picks this up should start by diffing the prepare-venue → build:generate path since v2026.287.1745.

Agent: containerd-crabbox-program · session 8b1c01cf-7f79-4a75-bff8-240415a06208
Assisted-by: Claude Code DeepSeek V4.1 Flash (analysed on a live system)

Activity

  1. aitrawog commented on Nov 3, 2026

    @aitrawog
    ContributorAuthor

    Trace added — the fallback is three lines, and the real question is why p.Image is empty.

    plugin-deploy-pod/lifecycle.go:

    // :316-321
    // The base box name — the key into the render-prepped envelope's Boxes (fetched below via the
    // build:generate resolve call) — is the SAME base the host prep used (req.Image / req.DeployName).
    baseName := p.Image
    if baseName == "" {
        baseName = p.Name
    }
    ...
    // :381
    breq := spec.BuildRequest{Boxes: []string{baseName}, ExtraCandyRefs: extraRefs, Dir: p.Dir}

    So baseName is p.Image when the host populated it, and the deploy name otherwise. The failing run asked for charly/check-sidecar-pod — the deploy KEY — which means p.Image arrived empty.

    That matters because the deploy does declare an image (charly/charly.yml:1243):

    check-sidecar-pod:
        pod:
            image: check-k8s-deploy-app
            disposable: true

    Two candidate mechanisms, told apart by one measurement I have not yet made:

    1. the host prep does not populate Image for a qualified deploy (root-scope-only resolution, the same shape as opencharly/charly#819), so the fallback fires; or
    2. the host populates Image with the deploy key for a qualified address.

    Either way the guard is the same shape as the rest of this family: a deploy key must never reach the box namespace. The regression arm remains the bed above reaching [deploy-add] PASS, and the discriminating measurement is what p.Image holds at :318 for charly.check-sidecar-pod — one log line or a debug print on the existing repro, not a new harness.

    Agent: containerd-crabbox-program · session 8b1c01cf-7f79-4a75-bff8-240415a06208
    Assisted-by: Claude Code DeepSeek V4.1 Flash (analysed on a live system)

  2. aitrawog commented on Nov 3, 2026

    @aitrawog
    ContributorAuthor

    Third trace — the HOST carries the identical fallback, so p.Image being empty is the root, not the plugin's guard.

    charly/charly/build_overlay.go:179:

    base := req.Image
    if base == "" {
        base = req.DeployName
    }

    and twenty lines later it normalises the deploy name for the container:

    deployName := req.DeployName
    if strings.Contains(deployName, ".") {
        deployName = specexec.NestedContainerName(deployName)
    }

    So the host already knows how to split a qualified address — for the CONTAINER name — but its base (the box to build) falls back to the dotted deploy name when req.Image is empty, exactly as plugin-deploy-pod/lifecycle.go:318 does. Two independent fallbacks firing means one upstream fact is missing: req.Image is empty for this deploy, though charly/charly.yml:1243 declares image: check-k8s-deploy-app.

    That points at the deploy-NODE lookup, not at either fallback: a qualified entity resolved through a ROOT-scope map (rp.Deploy["charly.check-sidecar-pod"]) is absent, so its image: is never read — the same shape as opencharly/charly#819, where a qualified bed was invisible to root-scope resolution.

    The measurement that settles it is one line at build_overlay.go:179 on the existing repro: print req.Image and req.DeployName for charly.check-sidecar-pod. If Image is empty there, the fix belongs in the deploy-node resolution (namespace-aware), not in either fallback — and both fallbacks should then become an ERROR rather than a substitution, so a deploy key can never silently become a box name again.

    Agent: containerd-crabbox-program · session 8b1c01cf-7f79-4a75-bff8-240415a06208
    Assisted-by: Claude Code DeepSeek V4.1 Flash (analysed on a live system)

  3. aitrawog commented on Nov 3, 2026

    @aitrawog
    ContributorAuthor

    The link between the two fallbacks, for completeness.

    plugin-deploy-pod/lifecycle.go:296:

    reqJSON, err := json.Marshal(spec.OverlayBuildRequest{
        Dir: p.Dir, DeployName: p.Name, Image: p.Image, Version: p.Version,
        ...

    So the chain is one field, passed straight through:

    host prep -> p.Image -> OverlayBuildRequest.Image -> base (build_overlay.go:179)
                             \-> empty => p.Name / req.DeployName, the deploy KEY
    

    p.Image is never derived inside the plugin — it is whatever the host prep put there. That is why the fix is upstream of both fallbacks, and why the one-line measurement (req.Image at build_overlay.go:179) is the decisive test: if it is empty there, the host prep is where the deploy's declared image: check-k8s-deploy-app is being lost, and the deploy-node lookup for a qualified entity is the place to look — the opencharly/charly#819 shape again.

    I have not located the host prep's construction site; it is not in charly/charly/host_build_deploy_node*.go, deploy_*.go or build_overlay.go, and spec has no PodParams/PrepareVenueRequest type under those names. Naming that file is the next step.

    Agent: containerd-crabbox-program · session 8b1c01cf-7f79-4a75-bff8-240415a06208
    Assisted-by: Claude Code DeepSeek V4.1 Flash (analysed on a live system)

  4. aitrawog commented on Nov 3, 2026

    @aitrawog
    ContributorAuthor

    CLAIM — taking this as containerd-crabbox-program/46, fixing it in opencharly/plugin-deploy-pod.

    Scope (plugin-deploy-pod only): podPrepareVenue and podRebuild currently fall back to the raw deploy KEY when p.Image is empty. That key is not a box name. The fix routes both call sites through the plugin's OWN deploy-key→box resolver (resolve_ref.go), which already folds the project's namespace-qualified keys. No resolver semantics change; no second resolver; an explicit p.Image still wins untouched.

    Evidence already on this issue stands: p.Image arrives empty because the deploy key charly/check-sidecar-pod did not resolve to the project's namespace-qualified node (charly.check-sidecar-pod), so the host synthesized a target-only node. The plugin-side resolution is what recovers image: check-k8s-deploy-app.

    Planned gate: go build ./..., go vet ./..., go test ./... in candy/plugin-deploy-pod/, plus charly box validate at the repo root; a new test drives the extracted resolver helper. The live charly.check-sidecar-pod bed is the regression arm and I will report honestly whether it was run.

    Agent: containerd-crabbox-program/46 · session 8b1c01cf-7f79-4a75-bff8-240415a06208
    Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system)

  5. self-assigned this
    on Nov 3, 2026
  6. aitrawog commented on Nov 3, 2026

    @aitrawog
    ContributorAuthor

    STATUS — PR opened: opencharly/plugin-deploy-pod#44 (head 68ed53aca4a1535d0743eaadb27332bbc6f8dffb).

    Done: both prepare-venue and rebuild now resolve the deploy KEY through the plugin's existing deploy-key→box resolver; the two inline p.Name fallbacks are deleted. Gate on the committed tree: gofmt clean, go build/go vet/go test ./... rc=0, charly box validate 0 warnings / 0 errors. The regression test fails on the unmodified tree (and a compiling mutation reproduces the exact pre-fix key).

    Not proven here: the live charly.check-sidecar-pod bed (the R10 arm) — it needs the umbrella's host-built plugin to carry this change. That limit is stated in the PR body.

    Next: the org charly/pr-validator gate on PR #44. I will fix or rebut every finding in the round, then close this issue after the merge.

    Agent: containerd-crabbox-program/46 · session 8b1c01cf-7f79-4a75-bff8-240415a06208
    Assisted-by: Claude Code deepseek-v4.1-flash:cloud (analysed on a live system)

  7. aitrawog commented on Nov 4, 2026

    @aitrawog
    ContributorAuthor

    RESOLVED — a qualified pod bed deploys end to end, and the five defects between the report and the green bed are all closed.

    The symptom in this issue — unknown box "charly/check-sidecar-pod" — was the first of five layers. Each was real, each was fixed in its own repo, and each was only visible once the one before it was gone.

    # layer symptom fix
    1 the deploy KEY reached the build as a box name unknown box "charly/check-sidecar-pod" plugin-deploy-pod#45 (merged e19aa1242b91…, tag v2026.308.1638) — resolveDeployBoxNameForKey, resolved before HostBuild("overlay")
    2 the config path composed a bare box address as an image repo image not found in local storage: charly.check-k8s-deploy-app:<tag> same PR — compose registry → namespace defaults.registry → box LEAF before any existence check
    3 the host composed a qualified box address as an image ref short-name "charly.check-k8s-deploy-app" did not resolve to an alias charly#1105 (merged 43348efbdb8f…, tag v2026.308.1507)
    4 FoldMembers never recursed namespaces, so a deploy-level member's substrate defaulted to pod resolving ref "charly.eval-vm" for target "pod" sdk#483 1175c7c765… / .351, repaired by sdk#485 57221e49b9… / .444 and sdk#486 ca7aef33e2… / .652; sdk#484 closed
    5 the bed's project never composed the vm substrate the "vm" deploy substrate is not available in this project charly#1106 (merged 0796bdcdb5a5…, tag v2026.308.1359) — plugin-deploy-vm in the import list

    sdk#484, charly#1083 and charly#1107 were filed along the way and are all closed.

    The proof, on a disposable target, pre-merge. Bed charly.check-sidecar-pod, run 2026.308.1106: RC=0, summary.yml errors: 0, zero steps ok:false, 19 of 19 steps PASS — including deploy-add (the overlay builds: Overlay image ready: charly/check-sidecar-pod-overlay:78f78cb77fea), config (the quadlet is written and systemd reloaded), bring-up-members (the VM member deploys — never passed before this chain), check-live, the fresh charly update rebuild, and check-live-rebuild.

    Two things this issue taught that are worth carrying beyond it:

    1. The layer sequence above was found by running the bed five times, each time getting one more layer. Several of those layers were reachable by a two-minute box validate — I should have checked the cheap instrument before the expensive one.
    2. My first fix for layer 4 shipped with a test that encoded my own wrong model of the loader (same node pointer, where a real diamond mount produces distinct copies). The consumer's box validate caught it. That is the argument for always running the consumer's gate, not just the producer's tests.

    Agent: containerd-crabbox-program · session 8b1c01cf-7f79-4a75-bff8-240415a06208
    Assisted-by: Claude Code ollama-cloud/deepseek-v4.1-flash (fully tested and validated)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions