Repository navigation
Podman port - #206
Open
rountree wants to merge 8 commits into
Open
Podman port#206rountree wants to merge 8 commits into
rountree wants to merge 8 commits into
Conversation
[podman-port] Reconstructs serial build
spindle-podman build serial
completed with
==========================================
Build complete!
==========================================
Image: spindle-serial-ubuntu
Next steps:
1. Run regular tests: ./scripts/podman/test-spindle-serial.sh
2. Run crash tests: ./scripts/podman/test-spindle-serial-crash.sh
3. Or manually: podman run --rm -it spindle-serial-ubuntu
✓ Build completed in 300s
⚠ Build time (300s) exceeded baseline (250s)
Copies from podman-port-old (serial build)
spindle-podman
common.sh
build-spindle-serial.sh
Dockerfile.podman
entrypoint.sh.podman
.dockerignore
Updates .gitignore with
.podman-state.json
.podman-metrics.log
.podman-logs
.*.swp
The check is here because of an earlier bug that kept adding build results to the same tarball; it eventually exceeded 96 GB. Slow build times were the first tip-off that something was amiss.
Added
test-spindle-serial.sh
Works:
ALL TESTS PASSED
==========================================
✓ All tests passed!
==========================================
Cleaning up container...
spindlenode
✓ Cleanup complete
✓ All tests passed in 68s
Working: spindle-podman build slurm-base
==========================================
Build complete!
==========================================
Image: spindle-slurm-base
Next step: Build the testing image
./scripts/podman/build-spindle-slurm-srun.sh
✓ Build completed in 580s
⚠ Build time (580s) exceeded baseline (450s)
Added
build-spindle-slurm-base.sh
Dockerfile.podman
build_mpich.sh.podman
Adds registry (targets.sh) describing each build target, expected completion time, etc. Allows for code duplication removal. Adds `--dry-run` to `spindle-podman build x` to allow verification that nothing in the build process has changed without needed a full build. Bugfixes: 1. logs no longer flags every later log in a session once one has errors 2. The grep -c ... || echo 0 pattern is fixed, so it can't write 0\n0 into the state JSON. 3. cleanup and defenestrate no longer delete `/tmp/spindle*` on the host. Adds: targets.sh
Adds `spindle-podman test slurm-srun`: a 4-worker Slurm cluster (MariaDB, slurmdbd, slurmctld) in rootless podman, running the Spindle testsuite (138 s end to end). New targets and commands: slurm-srun and mariadb (pulled) targets; save, load, env, diagnose; per-command help and typical workflows. Findings that shaped the design (details in STATE.md): - netavark starts aardvark-dns through `systemd-run --user`, which fails without /run/user/<uid>; the cluster helper starts aardvark-dns itself. - Spindle's configure picks its srun launch mode from `srun --version`, which fails without a slurm.conf; the image now ships a stub so Spindle gets --external-launcher. A build guard fails the image if it does not. - podman state follows XDG_RUNTIME_DIR, which moves when a login session appears; spindle-podman pins it. Also: hang detection with automatic snapshot (redacted), readiness polling instead of sleeps, per-target build baselines, pulled-image targets, and removal of the per-target build scripts.
…Slurm flavors sshd needs uid/gid 65534, which rootless podman does not map by default, so ssh logins were reset. The target adds --uidmap/--gidmap for 65534 to the Slurm containers (T_RUN_ARGS, from LLNL's container gotchas page). `diagnose ssh` shows it: login fails by default, works with the maps. Also: test-spindle-slurm.sh replaces the srun-only script; build guard shared as check_launch_mode.sh.podman (checks srun or rsh mode).
…dresses Adds `spindle-podman test slurm-plugin` (122 s): a Slurm cluster with Spindle loaded as a SPANK plugin. Spindle fix (src/slurm_plugin/plugin_utils.c): getHostAddrSinfo ran `sinfo -O NodeAddr`, whose column sinfo truncates to 20 characters. A node named slurm-plugin-1-node-1 came back as "slurm-plugin-1-node-", the front end could not resolve it, and the first test hung. Ask for `NodeAddr:255`. CI's names are short, so it never showed. (Fully qualified hostnames over 20 characters would hit it anywhere.) Podman side: - plugstack.conf is chmod 644: salloc/srun read it as the job user. - Cluster names for the plugin target use a short stem (T_CLUSTER_NAME) and the test script refuses names past 20 characters (T_NODE_NAME_MAX), so the bug cannot silently return; both lines can go once the fix is verified. - check_launch_mode.sh.podman gains a "plugin" mode (macro and installed libspindleslurm.so). Adds spindle-podman exec: runs podman exec with the pinned runtime directory, so it works from any shell.
This branch is waiting to be deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Work in progress.
Enables containerization similar to Docker+GitHub using rootless podman.