Tools for building, installing and maintaining an HPC software stack with EasyBuild on top of EESSI, the European Environment for Scientific Software Installations. They grew out of running a site software layer, and each one does one job that EESSI, EasyBuild or Slurm leave to the site. The READMEs of eb-local, eb-staged-rebuild, eessi-arch-guard and eb-keep-logs open with what EESSI, EasyBuild or Slurm already cover, and what the tool adds.
| Tool | Job | Docs | |
|---|---|---|---|
| eb-stack | Turn conda-forge, Spack, PyPI, CRAN and Cargo recipes into easyconfigs; solve, shard and plan an EasyBuild build list | docs | |
| eb-staged-rebuild | Rebuild an installation beside the live one and swap it in, for a stack outside CernVM-FS that jobs read during the rebuild | docs | |
| eessi-arch-guard | Reload EESSI in a Slurm task that inherited a stack loaded for another CPU or GPU, as srun from a login shell does | docs | |
| eb-local | Build on EESSI-extend inside a Slurm job's node-local disk, compile your own code with RPATH, list and uninstall per CPU target | docs | |
| eb-keep-logs | Keep a failed EasyBuild command's own output, which --tmp-logdir does not; rotate a pipeline's log tree | docs | |
| eb-pipeline | Drive a site's EasyBuild builds from one site file, through eb-stack, eb-staged-rebuild and eb-keep-logs | docs | |
| rfc9162 | Append-only Merkle logs hashed as RFC 9162, with tiled storage and signed checkpoints | docs | |
| zig-common | The shared Zig library under the Zig tools: system calls, checked files, child processes | docs |
The tools call each other through their command lines, so a site can adopt one without the rest.
- A user installs and runs software.
eb-localbuilds on EESSI-extend in a Slurm job's node-local disk, one tree per CPU target.eb-local env, which loads those modules in a new shell or job, runseessi-arch-guard --shellwhen the guard is installed, andeb-local doctorreports the target the guard selects. - A user submits a job.
eessi-arch-guard --check-submit --partition NAMEaskseb-stack stack archwhether every loaded module, from EESSI or from the user's own EESSI-extend tree, exists for that partition's CPU target, before anything is queued. As the SlurmTaskProlog, the guard then reloads a stack the task inherited from a node of another type. - A site learns what is used. The guard's Lmod site package can write
ModuleUsageTrackingrecords;eb-keep-logs usage --activeturns them into the installed software that was loaded, whicheb-stack stack gap --siteaccepts when planning the next stack. - A site builds and rebuilds.
eb-pipelinereads one site file and driveseb-stack stack shardandstack admitfor planning and inode admission,eb-staged-rebuild runfor a rebuild that jobs keep reading during, andeb-keep-logs failedandrotatefor the logs.eb-staged-rebuild run --ledger-configrecords each publication witheb-stack ledger append, andeb-stack ledger backfill,anchorandprojectmaintain that ledger, onrfc9162's tiled Merkle log.
eb-stack, eb-pipeline and rfc9162 are written in Rust.
The other tools are moving from Bash and Python to Zig, each still in its own repository, on a shared Zig library for running EasyBuild, keeping private files and logs, and locked state.
Each tool is then one static binary per CPU architecture.
That matters most for eessi-arch-guard, which runs once per Slurm task: its Python prolog takes 46 ms per task where the Zig binary takes 3.5 ms (tests/measure_noop.py, cold, on one workstation with the archdetect stub).
Each repository has its own README and documentation site, and each tool's release carries its binaries with checksums; zig-common, a library, is released as a tag.
The logos, the table above and the cross-links in every README and site come from branding/.