Repository navigation
feat(docker): preserve workspace storage during stopped image updates - #5617
senthilr-nv wants to merge 8 commits into
Conversation
Refs NVIDIA#5484. Keep workspace data separate from disposable runtime resources, validate retained volume identity, and preserve legacy layouts. Workload image replacement and native qualification remain separate work. Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Refs NVIDIA#5484. Preserve native identity and recorded data volumes while replacing workload bytes and refreshing container-bound runtime evidence. Keep update capability negotiation disabled pending lifecycle/restart integration and native qualification. Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Selected upstream target: e8ba30f Retain upstream shutdown grace and mutation-audit behavior alongside opt-in Docker workspace persistence and image-only configuration updates. Resolve the README overlap by documenting both the persistent-data contract and current shutdown behavior. Validation: formatting and 180 Docker driver unit tests passed; two live Docker tests remain ignored. Further lint and native integration qualification are recorded separately. Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Name the feature-selected registration array before iteration so Clippy does not flag the supported Docker-only build as a single-element loop. Driver entries, ordering and registration behavior remain unchanged. Validation: cargo fmt --all --check and Docker-only gateway Clippy with compute-driver-docker, telemetry and vendored-z3, all targets and warnings denied. Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
f0584ca to
8b9da39
Compare
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
johntmyers
left a comment
There was a problem hiding this comment.
gator-agent
PR Review Status
The Docker storage and stopped-image-update workflow fits accepted issue #5484. Two correctness problems need fixes before pipeline handoff: target identity compatibility is not checked, and an interrupted credential rotation can prevent subsequent Starts.
Action required: @senthilr-nv, address the two inline findings and add regression coverage for their failure paths.
Blocking findings:
- GATOR-8b9da399-01: Reject incompatible target workload identities before replacement.
- GATOR-8b9da399-02: Preserve Start retry recovery after interrupted credential rotation.
Carried findings: None.
Non-blocking suggestions:
- Update the Kubernetes-only update wording in skills/openshell-cli/SKILL.md and .agents/skills/helm-dev-environment/SKILL.md to reflect the restricted Docker workflow.
Gator metadata
- Validation: Implements accepted Docker-specific issue #5484 through the existing update contract.
- Docs: Docker configuration, runtime and sandbox update pages updated; existing navigation suffices.
- Checks: Existing current-head workflows succeeded; review was code-only and no local tests were run.
- E2E: Required for Docker lifecycle/storage changes; existing E2E jobs were skipped without test:e2e. Dispatch follows resolution of blocking review findings. Live A-to-B and interruption qualification remains an author-declared merge gate.
- Head SHA:
8b9da3993e232e484313adc2158d87284381bd9a - Base SHA:
e8ba30fdeff5f9bcc985b2c11064f0b0deb3c168 - Merge base SHA:
e8ba30fdeff5f9bcc985b2c11064f0b0deb3c168 - Patch ID:
3240105f250b3ea1397f7555296845db490815f3 - Gator payload: 13
- Review mode:
initial - Previous reviewed SHA: none
- Review budget exhausted: no
- Maintainer decision required: no
- Next state:
gator:in-review
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Compatibility review (advisory)New or updated breaking changes; see the full report for all findings. 1. api-behavior — No explicit risk acceptance foundDocker previously preserved spec.template.labels keys openshell.ai/workspace-storage and openshell.ai/workspace-volume-identity; it now strips them on create and interprets them as persistence metadata on existing containers.
Review details
Advisory review; publication gates are unchanged. |
|
Label |
PR Review StatusThanks @senthilr-nv. I checked your target-identity validation and interrupted credential-publication fixes against the previous review and the latest author delta. Both findings are fixed, their Gator threads are resolved, and no blocking code findings remain. The target preflight now rejects incompatible or unsupported identities before recording intent or deleting the source. Stopped Start checks each file's immutable bindings before authenticated repair, requires a fresh stopped-container inspection, and checks full consistency before staging. The new regression cases cover these paths; this follow-up was code-only, and I did not independently rerun the reported local tests. Required repository E2E has been enabled. The focused live driver test does not qualify the full gateway/supervisor path. Action required: @senthilr-nv, provide the remaining full gateway/supervisor Docker qualification evidence for authenticated workload use, session recovery across image replacement/restart, and end-to-end removal before merge. Blocking findings: No blocking code findings remain. Carried findings: None; GATOR-8b9da399-01 and GATOR-8b9da399-02 are fixed. Gator metadata
|
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
Compatibility review (advisory)New or updated breaking changes; see the full report for all findings. 1. api-behavior — No explicit risk acceptance foundDocker previously preserved caller labels
Review details
Advisory review; publication gates are unchanged. |
PR Review StatusThanks @senthilr-nv. I checked the new Docker-specific persistence labels, legacy-label regressions, owned-gateway wrapper changes and authenticated Docker qualification suite against the previously reviewed head. The independent follow-up review found no new blocking findings. Both earlier findings remain fixed and their threads remain resolved. The new suite covers authenticated workload access, image replacement, gateway restart, credential-publication repair and deletion. Adding the suite does not establish a successful live run: it requires the documented opt-in wrapper and is not selected by the standard E2E execution matrix. I checked the wrapper's shell syntax; no local Rust or live Docker tests were run in this sandbox. Action required: @senthilr-nv, provide a successful full gateway/supervisor Docker qualification result with the tested source revisions and input hashes before merge. Required repository checks must also pass on this head. Blocking findings: No blocking code findings remain. Carried findings: None; GATOR-8b9da399-01 and GATOR-8b9da399-02 remain resolved. Gator metadata
|
|
Full Docker gateway/supervisor qualification now passes locally on Environment: macOS 26.7.1 arm64; Docker 29.8.0; Rust 1.95.0. The owned test gateway used mutual TLS with certificate verification enabled. No existing gateway or Docker Desktop setting was changed. Verified through the public API and real CLI/SSH:
Command: Build provenance: gateway and CLI were rebuilt from
The compatibility advisory is also addressed in The final source is signed and DCO passed. Required Branch Checks and enabled repository E2E are still pending on this head; this local evidence does not waive them. Please review the compatibility delta and full qualification evidence before approval/merge. |
|
Compatibility follow-up: the latest advisory is valid. Renaming the persistence labels in I am moving persistence authority to a protected per-sandbox driver record, bound into the private runtime descriptor, while preserving all template labels. The follow-up will also gate volume admission and deletion on that record. Please hold merge until the fix, regression tests and repeated full qualification are posted. The current CI runs can continue. |
Signed-off-by: Senthil Ravichandran <senthilr@nvidia.com>
|
The compatibility follow-up is implemented and qualified in signed, DCO-signed-off commit Storage authority now comes from independent protected driver state. Validation on this head:
Command: Provenance: gateway, CLI, sandbox and supervisor all come from signed head
The final-head compatibility assessment is complete: no unaddressed breaks, zero findings, all nine required surfaces reviewed. It names source GitHub verifies all eight contribution signatures; DCO and accepted-issue checks passed. Final-head Branch Checks and Branch E2E Checks both passed, attempt 1. Both embedded/external Docker conformance and the Kubernetes/k3s-ha conformance lanes passed. The prior head's Kubernetes k3s-ha workspace test passed its functional assertions but failed API-delete cleanup after a 120-second timeout; that run is not a pass. The required final-head run is now green; the earlier failure is retained as historical evidence. The review follow-up reports no blocking obligations remaining. Final maintainer handoff/approval remains; no merge has been performed. |
PR Review StatusThanks @senthilr-nv. I checked your protected storage-authority delta and current-head Docker qualification evidence. Workspace labels now remain caller data; the private ownership record governs admission, update replay and removal. The independent critical-only follow-up found no new Critical defects. Both earlier findings remain fixed and their threads remain resolved. The qualification suite covers authenticated API/SSH use, retained workspace and SSH identity through image replacement and gateway restart, interrupted credential-publication repair, and API deletion. GitHub artifact metadata confirms that the cited sandbox and supervisor binaries came from this exact head. I inspected the suite and provenance; I did not independently rerun the reported live qualification. No blocking review obligations remain. Required Branch Checks and repository E2E are still running; Gator will inspect their results before a maintainer handoff. Carried findings: None; GATOR-8b9da399-01 and GATOR-8b9da399-02 remain resolved. Gator metadata
|
|
@johntmyers Both required runs are now green on |
Summary
Add opt-in persistent
/sandboxstorage and image-only updates for stopped Docker sandboxes through the existingSandboxUpdates.UpdateSandboxcontract. Replacing the workload container preserves the native sandbox identity and its owned workspace volume; Start remains a separate operation.Related Issue
Closes #5484
Changes
persist_workspace = falseto the Docker driver configuration. Newly created default/sandboxworkspaces can use driver-owned local volumes; existing writable-layer workspaces are not migrated.workspace-storage.json, bound into both private bootstrap and runtime descriptor. Recheck volume identity, namespace and workload UID/GID even when external resource admission is disabled. Preserve all caller template labels; labels cannot authorize persistent storage. Failed creation retains the ownership record, and replay rejects missing/changed authority before mutation. Stop retains data; explicit Delete removes admitted owned storage and reports failures.USER UID:GIDmatching the recorded identity; reject symbolic and implicit identity representations before saving intent or deleting the source. Reject command/environment changes and incompatible layouts before replacement.Release Notes
Docker gateways can opt new workspaces into persistent
/sandboxstorage and update their stopped workload image without changing the OpenShell sandbox ID. The gateway must separately enable sandbox updates. Existing writable-layer workspaces remain unchanged.Testing
Current local validation uses Rust 1.95.0 on macOS arm64. This branch incorporates main at
e8ba30fdeff5f9bcc985b2c11064f0b0deb3c168.cargo fmt --all --checkcargo test --locked -p openshell-driver-docker --lib: 190 passed, 0 failed, 3 ignored. The default suite uses mock Docker endpoints; live tests require explicit opt-in.cargo clippy --locked -p openshell-driver-docker --all-targets -- -D warningscargo clippy --locked -p openshell-gateway --no-default-features --features compute-driver-docker,telemetry,vendored-z3 -- -D warningsdocker-updatessuite clippy with--all-targets -- -D warnings.USERaccepted, and Start rejected interrupted bootstrap/descriptor publication before repair.cargo test --locked -p openshell-driver-docker --lib live_stopped_image_replacement -- --ignored --nocapture: 1 passed on Docker 29.8.0 using the committed Alpine fixture. Verified incompatible numeric/symbolic refusal, same sandbox/volume and retained file bytes, A-to-B image contents, settled replay and recovery after source deletion. This is driver integration, not gateway/supervisor E2E.0acdea553: run44005057843, attempt 1.0acdea553: mTLS client rejection/authorization, real API and SSH execution on A/B images, retained sandbox/SSH/workspace identity, gateway restart and Stop/Start, interrupted credential-publication repair through Start, and API removal of the workload, all four owned volumes and private runtime state.docker-updates: 1 passed, 0 failed, 0 ignored. Gateway/CLI built from this head; Linux sandbox/supervisor artifacts from this exact head’s CI run44005058200and packaged with the official Dockerfiles. See the qualification comment for commands and hashes.0acdea553: run44005058200, attempt 1, including both Docker conformance lanes and Kubernetes/k3s-ha conformance. The prior-head cleanup timeout did not recur.Checklist