Skip to content

Latest commit

 

History

History
559 lines (453 loc) · 55.1 KB

cli.org

File metadata and controls

559 lines (453 loc) · 55.1 KB

The eb-pipeline command

The eb-pipeline command

eb-pipeline reads a site file and a build list, and plans or runs the phases that build the list on one target of the site. Each command has its own options; docs/examples/cli.sh prints every help text on this page.

Synopsis

eb-pipeline <COMMAND>
eb-pipeline check [OPTIONS]
eb-pipeline schema
eb-pipeline init [OPTIONS]
eb-pipeline explain [OPTIONS] --site <SITE>
eb-pipeline targets [OPTIONS] --site <SITE>
eb-pipeline status [OPTIONS] --site <SITE>
eb-pipeline metrics [OPTIONS] --result <RESULT>
eb-pipeline plan [OPTIONS] --site <SITE> --target <TARGET> --build-type <BUILD_TYPE>
eb-pipeline run [OPTIONS] --site <SITE> --target <TARGET> --build-type <BUILD_TYPE>
eb-pipeline verify [OPTIONS] --site <SITE> --target <TARGET> --build-type <BUILD_TYPE> --from <FROM>
eb-pipeline watch [OPTIONS] --site <SITE> --target <TARGET> --build-type <BUILD_TYPE>
eb-pipeline audit [OPTIONS] --site <SITE> --target <TARGET> --out-dir <OUT_DIR>

eb-pipeline --help prints:

Site-neutral EasyBuild pipeline driver configured by a per-site TOML file

Usage: eb-pipeline <COMMAND>

Commands:
  check    Parse and validate a site file
  schema   Print the site file reference: every key, type, default and meaning
  init     Write a starter site file from a built-in template
  explain  Print the resolved configuration of one or all targets
  targets  Print the target names in file order, for a CI matrix or a shell loop
  status   Print the last runs under the log root, newest first, from their summaries
  metrics  Print Prometheus text, or a short JSON status, for one result file
  plan     Compute phases, launch lines, environments and tool calls without executing
  run      Execute the plan's phases as subprocesses
  verify   Recompute the input record and check its inclusion proof
  watch    Rebuild the recorded installs an EESSI change reaches, in dependency order
  audit    Run eb-stack vulnerability checks and collect rebuild lists
  help     Print this message or the help of the given subcommand(s)

Options:
  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version

Examples:
  eb-pipeline init --template small-cluster --out site.toml
  eb-pipeline check --site site.toml
  eb-pipeline check --list stack.yml --check-from-commit --easyconfigs-repo DIR
  eb-pipeline check --site site.toml --gap-out DIR --generation NAME --shadow
  eb-pipeline targets --site site.toml --format json
  eb-pipeline plan --site site.toml --list list.txt --target cpu --build-type production --out plan.json
  eb-pipeline run --site site.toml --list list.txt --target cpu --build-type production --result result.json
  eb-pipeline watch --site site.toml --target cpu --build-type production --dry-run
  eb-pipeline audit --site site.toml --target cpu --findings findings.csv --out-dir audit
  eb-pipeline metrics --result result.json

Companion commands from this package:
  eb-retry-list
  eb-shard-weights
  eb-test-map
  eb-junit
  eb-cvmfs

Configuration: every flag above falls back to an EB_PIPELINE_<FLAG> variable
(EB_PIPELINE_SITE, EB_PIPELINE_LIST, EB_PIPELINE_PACKAGE_PLAN,
EB_PIPELINE_TARGET, EB_PIPELINE_BUILD_TYPE, EB_PIPELINE_RUN_ID,
EB_PIPELINE_LOG_STAMP, EB_PIPELINE_LISTS_DIR, EB_PIPELINE_RESULT); a flag on
the command line wins.
EB_PIPELINE__SECTION__KEY=value sets one site file value, for example
EB_PIPELINE__FILESYSTEMS__SCRATCH=/scratch/eb; the result is validated like the
file.

Output streams: stdout carries plan, explain, schema, init, status, targets and
metrics, and verify, and run --dry-run or watch --dry-run output only. Events go to stderr
as `LEVEL [phase] target: message`. The result JSON goes to --result, a run's
requested plan JSON to --plan-out, and each run's summary to summary.json and
summary.html in its run directory,
<log_root>/<log-stamp>/. Run --metrics-file atomically replaces a Prometheus
text file when the run ends. `metrics` exits 0 when it can read the file,
including when the run itself failed.

Exit codes:
   0  all requested work completed
   1  work attempted and failed (a build line or a phase); see the result file
   2  usage error, invalid site file, or a tool below its minimum version
  75  a resource is temporarily unavailable or a step is pending; rerun
  78  refused by a precondition, by admission or by eb-staged-rebuild; lines
      built before the refusal stay built, see the result file
 130  interrupted by SIGINT, after the running child ended
 143  interrupted by SIGTERM, after the running child ended

Sibling tool codes are mapped explicitly:
  eb-stack stack admit
                    0 admitted, 78 refused, 1 failed, 2 usage
  eb-stack stack shard
                    the code in shards.planner_exit_unsharded (default 3) means
                    unsharded; the run continues
  eb-staged-rebuild 2 is a refusal there; the driver records it as refused
                    (exit 78), 75 as pending (exit 75), any other non-zero as failed
  eb-keep-logs      rotate 75 means another rotate holds the log root's lock;
                    the logs phase is recorded as skipped, any other non-zero
                    as failed
A signal (130, 143) or a usage error (2) ends the run with that code. Otherwise,
when several apply: 1, then 78, then 75, then 0. Codes are never summed.

eb-pipeline --version prints the version of the crate, the same for a release and for a build from source:

eb-pipeline 0.3.1

Arguments

The first argument names the command. No command takes an argument other than its options.

CommandWhat it does
checkRead the site file, check it against every rule of The site file, and print warnings about settings that are valid but probably wrong.
schemaPrint every key of the site file with its type, its default and what it does, section by section.
initPrint a built-in template site file, or write it to a new file.
explainPrint what each target resolves to: paths, launch lines and the environment of its commands.
targetsPrint the names of the targets in the order of the site file.
statusPrint the last runs under the log root, newest first, from the summary of each run directory.
planPrint the plan of a run as JavaScript Object Notation (JSON): its phases and every command with its launch line and environment. It starts nothing.
runRun the phases of the plan, one command at a time, write the result file, and add the run to the summary of its run directory.
verifyRecompute the input record of a plan or a result and check its RFC 9162 inclusion proof. Needs [attest] enabled.
watchCompare what each recorded install of a target was built against in EESSI with the EESSI that its module environment loads now, and rebuild the installs that a change reaches, in the order of their dependencies.
auditRun eb-stack vulnerability subcommands for one target and copy any generated rebuild lists to an output directory for eb-pipeline run --list.
helpPrint the help of the program, or of the command it names.

Two hidden packaging subcommands (completions and man) emit shell completion scripts and the section 1 manual page from the CLI parser.

Options

Every command takes -h, which prints a summary of its options, and --help, which prints them in full; both exit 0. eb-pipeline -V and eb-pipeline --version print the version. An option takes its value as the next argument or after an equals sign, as --site site.toml or --site=site.toml. An option with a variable in the table takes its value from that variable when the command line does not give it.

check

OptionVariableMeaning
--site FILEEB_PIPELINE_SITEThe site file. Required unless --check-from-commit is set.
--lists-dir DIREB_PIPELINE_LISTS_DIRA directory that must contain every file that the build_lists of a target names. A missing file is an error of the site file.
--probe-toolsStart BIN --version for each sibling tool that an enabled phase calls, and fail when one cannot start, prints no version, or prints one below the min_version of its [site.tools] table.
--list FILEEB_PIPELINE_LISTThe build list or easystack passed to --stack-checks or --check-from-commit.
--target NAMEEB_PIPELINE_TARGETUse only this target’s cpu_target and expanded source paths. Requires --stack-checks. Without it, target-dependent values from every target are used.
--stack-checks LISTComma-separated files, arch, cuda-cc, generation, provided or known-failures. Each names an eb-stack stack subprocess. Requires --list and --site.
--check-from-commitRun eb-stack stack from-commit --json on an easystack --list; off by default. Requires --list.
--easyconfigs-repo DIRPassed to eb-stack when the easystack has from-commit pins.
--easyblocks-repo DIRPassed to eb-stack when the easystack has include-easyblocks-from-commit pins.
--easybuild-version X.Y.ZEasyBuild release version; overrides eb-X.Y.Z in the list file name.
--gap-out DIRRun eb-stack stack gap and write its report files into DIR. Requires --generation. Without it, no gap or shadow subprocess starts.
--generation NAMEToolchain generation passed to stack gap, such as 2026.1. Requires --gap-out.
--shadowAlso run eb-stack stack shadow --json, write its JavaScript Object Notation (JSON) output to DIR/shadow.json, and preserve its failure when a local recipe shadows the base stack. Requires --gap-out.

check writes INFO [check] NAME: site file is valid to stderr when the file is valid, where NAME is site.name. When --check-from-commit runs, eb-pipeline forwards eb-stack’s report and exits with its status; exit 1 means at least one pin is already in the release tag. See Check from-commit pins before building. Before that line it writes a WARN line for each of these settings:

  • modules.unuse_path with a first modules.init entry that is not an absolute path;
  • admission.required without admission.quota_cmd and admission.reservations, so that run refuses with exit 78;
  • admission.quota_cmd and admission.reservations without admission.live_jobs_cmd.

Without --stack-checks, check starts no checks and otherwise behaves as before. With it, check runs every selected command directly, without scheduler.launch:

Nameeb-stack command and site values
filesstack files --easystack FILE; adds [shards].easyconfigs, filesystems.configs_repo and the selected targets’ expanded filesystems.source_paths.
archstack arch --easystack FILE; adds [shards].easyconfigs and the selected targets’ cpu_target values as --arch.
cuda-ccstack cuda-cc --easystack FILE; adds [shards].easyconfigs. eb-stack checks its bundled accelerator targets.
generationstack generation --easystack FILE.
providedstack provided --build-list FILE --strict; adds [sbom].lock as --baseline-lock when set. It expects a plain build list, and eb-stack refuses when no baseline is configured.
known-failuresstack known-failures --easystack FILE --strict; adds [shards].easyconfigs and the selected targets’ cpu_target values as --arch.

Repeated path and architecture values are passed once. The configured [site.tools.eb_stack].bin is used, or eb-stack on the executable search path (PATH). All requested subprocesses run even after one fails. If one or more exit nonzero or cannot start, check reports each failure and combines their statuses with the same signal, usage, failure, refusal and temporary-failure priority as a run summary. A command that cannot start contributes exit 1.

With --gap-out, check passes every target’s resolved modules_root/all as stack gap --site. It takes European Environment for Scientific Software Installations (EESSI) base module trees from shards.installed_modules_dirs, the variables named by shards.installed_modules_dirs_from_env, and watch.modules_dirs. It takes recipe trees from shards.easyconfigs and watch.easyconfigs. Repeated paths are passed once. The configured [site.tools.eb_stack].bin is used, or eb-stack on the executable search path (PATH).

stack gap receives the configured trees as repeated --site, --installed and --easyconfigs options, then --generation and --out-dir. Its own report files are written directly into DIR. eb-pipeline writes its standard output to gap.stdout.log and its standard error to gap.stderr.log. It also prints the summary line without changes. With --shadow, stack shadow receives the recipe trees as --easyconfigs and the EESSI trees as --base, with JavaScript Object Notation (JSON) selected by --json. Its standard output is DIR/shadow.json and its standard error is DIR/shadow.stderr.log. Each complete stream file atomically replaces an existing file. Neither subprocess uses scheduler.launch.

schema

schema takes no options. It prints the reference of the site file on stdout, one block per section, with one entry per key: the key, its type, whether it is required or its default, and what it does.

init

OptionVariableMeaning
--template NAMEThe template to print; required unless --list is given.
--listPrint the name and summary of each template, one per line, and nothing else.
--out FILEWrite the template to FILE instead of stdout, and write INFO [init] -: wrote template NAME to FILE to stderr. - is stdout. An existing FILE is refused, with exit 2.
--hintsEB_PIPELINE_INIT_HINTSAfter writing a file, print next steps on stderr; also enabled when EB_PIPELINE_INIT_HINTS=1.

The site file templates shows each template.

explain

OptionVariableMeaning
--site FILEEB_PIPELINE_SITEThe site file; required.
--target NAMEEB_PIPELINE_TARGETThe target to explain; every target when absent.
--build-type NAMEEB_PIPELINE_BUILD_TYPEApply the [targets.build_types.NAME] table of each target to what it prints.
--format FORMATtoml, the default, prints Tom’s Obvious Minimal Language (TOML); json prints the same as JSON.

For each target, explain prints the resolved paths, the launch line and arguments of each phase role, and the environment of its commands. It shows <run-id> and <log-stamp> where a run puts its own.

targets

OptionVariableMeaning
--site FILEEB_PIPELINE_SITEThe site file; required.
--format FORMATtext, the default, prints one name per line; json prints a JSON array of the names on one line.

status

OptionVariableMeaning
--site FILEEB_PIPELINE_SITEThe site file; required.
--target NAMEEB_PIPELINE_TARGETOnly the runs that have a log directory for the target, with only its entry; every run when absent.
--last NHow many matching runs to print, from 1 to 65535; 10 when absent. Other values exit 2.
--problems-onlyKeep failed runs and runs without a readable summary.json; keep every run when absent.
--format FORMATtext, the default, prints a line per run and a line per target; json prints one JSON document.

status reads the directories under filesystems.log_root whose names are log stamps, newest first, and leaves out every other entry. With --problems-only, --last counts runs after filtering. With --target, the selected target’s exit code decides whether a run is problematic. The run summary shows what it prints.

metrics

OptionVariableMeaning
--result FILEEB_PIPELINE_RESULTThe result file of one run; required.
--format FORMATprometheus, the default, prints Prometheus text; json prints one JSON status document.

metrics reads that file and prints these gauges:

  • eb_pipeline_info
  • eb_pipeline_exit_code
  • eb_pipeline_last_run_timestamp_seconds
  • eb_pipeline_last_run_result
  • eb_pipeline_last_run_duration_seconds
  • eb_pipeline_items_built
  • eb_pipeline_items_failed
  • eb_pipeline_lines
  • eb_pipeline_phase

A status the run did not use is 0. Log paths stay in the result file. Exit 0 means the file was read, including when the run’s own exit code is not 0. Exit 2 means the file is missing, is not JSON, or its kind is not result.

plan

OptionVariableMeaning
--site FILEEB_PIPELINE_SITEThe site file; required.
--list FILEEB_PIPELINE_LISTThe build list, or an easystack whose name ends in .yml or .yaml; required unless --package-plan is present.
--package-plan DIREB_PIPELINE_PACKAGE_PLANAn eb-stack package plan bundle. Its emitted recipes replace the build list. Mutually exclusive with --list.
--verify-package-planAdd ecosystem-native verification after the package-plan build. Requires --package-plan.
--target NAMEEB_PIPELINE_TARGETThe target to build on; required.
--build-type NAMEEB_PIPELINE_BUILD_TYPEThe build type; required. A name without a [targets.build_types.NAME] table takes the values of the target.
--run-id IDEB_PIPELINE_RUN_IDThe run label passed to the tools; <run-id> when absent.
--log-stamp STAMPEB_PIPELINE_LOG_STAMPThe name of the run’s directory under the log root, a Coordinated Universal Time (UTC) as YYYYMMDD_HHMMSS; <log-stamp> when absent. Any other value is refused, with exit 2.
--out FILEAtomically replace FILE with the complete plan instead of writing stdout; - is stdout. FILE must differ from the site and list files.
--shard-plan DIRRead the non-empty units from DIR/plan.json, written by eb-stack stack shard, without planning another shard-planner command.
--offline-bundle DIRAdd the opt-in offline planner calls to the sources phase and name the bundle that run writes. Needs an active [sources] table, a source path and configured easyconfig trees.
--require-offlinePlace sources.offline_wrapper before admission, build, check, cache, log and notification commands. Requires --offline-bundle and a non-empty wrapper.

The --list or --package-plan path must be valid UTF-8 because the plan and result record it as a JSON string.

An easystack uses the same YAML anchors, aliases and << merge keys as EasyBuild. Comments do not become data, an empty options mapping adds no arguments, duplicate entries stay in file order, and top-level keys other than easyconfigs are ignored. Each easyconfig entry may contain only its options mapping after YAML merges are applied. Syntax, duplicate-key and encoding errors name the input as FILE:LINE:COLUMN. A duplicate-key error points to the repeated key in its own mapping. UTF-8 byte-order marks and CRLF line endings are accepted.

run

run takes the common input and offline options of plan, and these:

OptionMeaning
--result FILEAtomically replace FILE with the complete result when the run ends. Without it, no result file is written. An input error leaves an existing FILE as it was. It must differ from --metrics-file.
--plan-out FILEAtomically replace FILE with the plan computed by this invocation before any phase command starts. A dry run writes it too. FILE must differ from every result and metrics path.
--metrics-file FILEAtomically replace FILE with Prometheus text when the run ends. Without it, no metrics file is written. A dry run does not write it. It must differ from --result and --resume-result.
--resume-result FILEContinue from FILE when [resume] enabled is true and the input digest still matches. Conflicts with --dry-run. Without that, exit 2 and no result file. A mismatch, a missing input_digest, or an incomplete identity exits 78 and writes the result.
--shard-plan DIRRead the non-empty units from a plan previously written by eb-stack stack shard. The run, its --plan-out document and its dry-run output use those units without a shard-planner command.
--dry-runPrint each command of a fresh plan on stdout and start nothing. Conflicts with --resume-result. No result file, metrics file or summary is written, even with an output option.
--oom-preflightBefore any package starts, ask eb-stack stack oom about the planned packages. Requires --node. Without this option no out-of-memory check runs.
--node NODEThe node passed to eb-stack stack oom, as CORES:MEMORY_MIB or CORES:MEMORY_MIB:TARGET. Requires --oom-preflight.

--run-id defaults to the log stamp, and --log-stamp to the UTC time at which the run starts, taken once. Every explicit run output must differ from the site and list files. A metrics file records the final result, end time and duration after post-run work such as Software Bill of Materials (SBOM) generation. If the plan file cannot be written, run exits 1 before starting a phase command. When both output files are requested, run attempts --result first and --metrics-file second even if the first write fails, then reports every output failure and exits 1. If --metrics-file resolves to the same file as either JSON option, run exits 2 before it starts work or changes any of those files. --result may name the same file as --resume-result: the complete prior result is read before the completed run atomically replaces it. A line that eb-staged-rebuild builds passes RUN_ID-STEM as its run id, where STEM is the easyconfig file name without .eb, with .2, .3 and so on added when the list repeats the line. When the OOM count is greater than zero, the preflight is refused, every planned package is recorded as skipped, and the run exits 78. The command reads only the oom=N summary line from eb-stack; eb-pipeline does not estimate memory itself.

With --dry-run, a planned command prints as [PHASE] COMMAND and a phase that starts nothing as # [PHASE] STATUS. The preview does not evaluate a prior result, so --dry-run and --resume-result together exit 2.

verify

verify takes --site, --target and --build-type as plan does, and these:

OptionVariableMeaning
--list FILEEB_PIPELINE_LISTRecompute from this build list or easystack. Use this or --recorded-list.
--recorded-listRecompute from the list path in --from. Conflicts with --list.
--from FILEThe plan or result JSON that holds input_attestation. Required. Needs [attest] enabled.

verify recomputes the input record and checks the inclusion proof in FILE. A relative recorded list path is resolved from the current working directory. A record that no longer matches the inputs exits 78. A proof that rfc9162-sidecar verify-proof rejects exits 1. Without [attest] enabled, verify exits 2. Attest the inputs of a plan is the task.

watch

OptionVariableMeaning
--site FILEEB_PIPELINE_SITEThe site file; required. It needs a [watch] table.
--target NAMEEB_PIPELINE_TARGETThe target whose record to compare; required.
--build-type NAMEEB_PIPELINE_BUILD_TYPEThe build type of the rebuild; required.
--run-id IDEB_PIPELINE_RUN_IDThe run id of the rebuild; the log stamp when absent.
--log-stamp STAMPEB_PIPELINE_LOG_STAMPThe run directory of the rebuild under the log root, as for run; the UTC time at which watch starts when absent. Any other value is refused, with exit 2.
--dry-runPrint the plan on stdout, and rebuild and record nothing.
--adoptRecord each install in the module tree of the target against the EESSI loaded now, in place of the record, and rebuild nothing.
--plan-out FILEAtomically replace FILE with the complete plan in JSON, including assessed actions when rebuild ordering stops before an order is computed.
--result FILEAtomically replace FILE with the complete rebuild result. Without it, or without a rebuild, no result file is written.

The plan and result destinations must differ from the site file and from each other. Without --dry-run, watch prints the plan to stderr, then runs the rebuild list through the phases of the site file, as run runs a build list. The watch state and plan says what watch compares, what its plan holds and how it rebuilds.

audit

OptionVariableMeaning
--site FILEEB_PIPELINE_SITEThe site file; required.
--target NAMEEB_PIPELINE_TARGETThe target to audit; required.
--findings FILEVulnerability scanner Comma-Separated Values (CSV) export for eb-stack rebuild-set --findings. Mutually exclusive with --cve.
--cve IDCommon Vulnerabilities and Exposures (CVE) id to pass to eb-stack stack sbom --cve and eb-stack component-reach --cve. Repeatable. Mutually exclusive with --findings.
--strictExit non-zero when eb-stack reports any backport item in its summary output.
--out-dir DIRDirectory where audit artifacts are written, including copied rebuild lists under DIR/rebuilds/*.yml. Required.

audit starts eb-stack as a subprocess and does not implement vulnerability matching itself. With --findings, it runs eb-stack rebuild-set --findings and copies the generated files from rebuilds/ into --out-dir/rebuilds/. With --cve, it runs eb-stack stack sbom --cve --backport-list and eb-stack component-reach --cve. When --strict is passed and eb-stack reports backport items, audit exits 1.

Environment

Each option in the tables above that names a variable reads it when the option is not on the command line. EB_PIPELINE__SECTION__KEY=VALUE sets one value of the site file, and a run sets variables of its own for the commands it starts. Environment variables lists all of them.

Output pipelines

plan and run --dry-run exit successfully when a pipeline consumer closes standard output after reading enough data. Another standard-output write error exits 1.

Exit status

StatusMeaning
0All requested work completed. watch: also nothing to rebuild.
1run: a line or a phase failed. watch: the rebuild failed, an eb-stack call failed, or the record, state file, plan or rebuild list could not be read or written. plan: the shard plan could not be read. status: the log root could not be read. Any command: an output file could not be written.
2A usage error, an invalid site file, an unknown target, a list that cannot be read, is empty or has a quote that is not closed, an easystack that gives no lines as written, a tool below its min_version, or an init --out file that cannot be created.
75run: a resource was busy or a step is pending; run it again. watch: another watch of the target runs, its module environment did not load, or EESSI cannot be read. audit: the module environment did not load, or EESSI cannot be read.
78run: a precondition, admission or eb-staged-rebuild refused. watch: no [watch], no EESSI, another EESSI version, no eb-stack, or no easyconfig tree for a vanished module. audit: no eb-stack, or no EESSI in the module environment.
130run and watch: interrupted by SIGINT.
143run and watch: interrupted by SIGTERM.

The commands other than run, watch and audit exit 0, 1 or 2. The rebuild of watch exits as a run would, and watch exits with its status. Exit status has the cases behind each status, how run picks one when several apply, and how it reads the exit statuses of the tools it starts.

Files

FileUse
the site file, --siteEverything about the site; The site file lists its keys.
the build list, --listOne easyconfig per line, followed by the options eb gets for it, as Name-version-toolchain.eb [eb options]. Blank lines and lines that start with # are skipped. A .yml or .yaml file is an EasyBuild easystack instead, each of whose easyconfigs entries is a line, as Build what eb-stack plans shows.
the plan, plan --outWhat plan prints; The plan and result files lists its keys.
the result file, run --resultWhat the run did, line by line and phase by phase; The plan and result files lists its keys.
DIR/manifest.json, --offline-bundle DIRVersion-one checksum index of the bundled files: url, sha256, local_name, consumer and the content-addressed object path.
DIR/objects/sha256/Bundle objects, stored once by SHA-256.
DIR/planner/stack order and stack files artifacts, plus one foreign-NN package-inspection bundle per [[sources.foreign]] input.
the metrics file, run --metrics-filePrometheus text for the completed run, atomically replaced for a textfile collector.
LOG_ROOT/LOG_STAMP/TARGET/The log directory of a run, under filesystems.log_root; The log tree and its locks lists its files.
LOG_ROOT/LOG_STAMP/summary.jsonThe summary of the runs with that log stamp, which each run updates and status reads, with summary.html beside it; The run summary lists its keys.
STATE_DIR/TARGET.jsonWith [watch], what each install of the target that a run or watch built was built against in EESSI, which watch compares with EESSI now; The watch state and plan lists its keys.
the plan, watch --plan-outWhat watch found for each recorded install and the list it rebuilds; The watch state and plan lists its keys.
DIR/rebuilds/*.ymlRebuild lists copied from eb-stack by audit --out-dir DIR, ready for eb-pipeline run --list.
DIR, check --gap-out DIRFiles written by eb-stack stack gap, its streams in gap.stdout.log and gap.stderr.log, and, with --shadow, shadow.json and shadow.stderr.log.

A list line is split into words as a shell splits them, with single and double quotes.

Companion tools

eb-retry-list

eb-retry-list reads a run result (--result) or one target of a run summary (--summary with --target) together with the original build list or easystack, named by --list or its recorded path, and writes the unfinished lines in their original order with their eb options.

eb-retry-list (--result FILE | --summary FILE --target NAME)
              (--list FILE | --recorded-list)
              [--status STATUS,...] [--classify] [--class CLASS,...]
              [--node CORES:MEMORY_MIB[:TARGET]] [--drop-oom]
              [--easyconfigs DIR]... [--arch ARCH]... [--out FILE]
              [--require-lines]
OptionMeaning
--result FILEResult file written by eb-pipeline run --result. Mutually exclusive with --summary.
--summary FILERun summary file (summary.json). Requires --target.
--target NAMETarget entry to read from --summary.
--list FILEBuild list or easystack (.yml or .yaml) passed to the run. Mutually exclusive with --recorded-list.
--recorded-listRead the build-list path from the result or selected summary target and use it as written. Mutually exclusive with --list.
--status STATUS,...Comma-separated statuses to keep: failed, refused, pending, skipped, unreached. Default: all non-=success= lines and unreached lines on a non-zero exit, omitting skipped lines whose note starts with resumed: unless --status skipped is passed.
--classifyPass each selected line’s recorded log to eb-stack stack bot-classify --json. Without --class, keep the interrupted, resource and timeout sub-classes.
--class CLASS,...Classes to keep instead of that default. A line stays when the JSON class or sub_class is one of these names. Repeatable. Requires --classify.
--node NODECORES:MEMORY_MIB or CORES:MEMORY_MIB:TARGET, passed to eb-stack stack oom --node. The lines that remain are written to a temporary easystack with their eb options and passed as --easystack. A line the command warns may run out of memory is named on stderr and still written. Requires --classify.
--drop-oomLeave out every line --node warned about. Requires --node.
--easyconfigs DIRRobot tree passed through to eb-stack stack oom --easyconfigs. Repeatable. Requires --node.
--arch ARCHCPU target passed through to eb-stack stack oom --arch. Repeatable. Requires --node.
--out FILEAtomically replace FILE with the complete retry list; - or absent prints to stdout. It must differ from the result or summary and from the explicit or recorded build list, including through symbolic and hard links.
--require-linesExit 1 after writing an empty retry list. Without it, an empty retry list exits 0.

--result, --list and --target fall back to EB_PIPELINE_RESULT, EB_PIPELINE_LIST and EB_PIPELINE_TARGET. A command-line option wins. --summary selects the summary instead of EB_PIPELINE_RESULT, and --recorded-list selects the recorded path instead of EB_PIPELINE_LIST.

Exit 0 means the retry list was written (empty when every line succeeded, was deselected, was not in a selected class, or was dropped by --drop-oom). With --require-lines, exit 1 means the retry list was written but is empty. Exit 2 is a missing or invalid input, an output path that aliases an input, an absent or unreadable recorded list, a selected line without a recorded log, an eb-stack that cannot start or fails, invalid classifier output, or stack oom output that has no single oom=N line matching its warnings. Classification and the memory check never fall back to retaining a line that eb-stack did not accept. The memory check refuses option forms that an easystack cannot represent and refuses one easyconfig with different option sets. eb-retry-list does not classify a log or estimate memory itself. When stack oom rejects its generated easystack, the error names the source build list instead of the deleted temporary file.

eb-shard-weights

eb-shard-weights turns observed duration_seconds of succeeded build lines from result.json, summary.json or --log-root into the MODULE<TAB>SECONDS table read by eb-stack stack shard --weights and [shards] weights. Each succeeded build line in a result must contain a non-negative numeric duration_seconds.

eb-shard-weights (--result FILE... | --summary FILE... | --log-root DIR [--last N])
                 [--target NAME] [--existing FILE] [--min-samples N]
                 [--mode max|latest|mean] [--out FILE]
OptionMeaning
--result FILEResult file written by eb-pipeline run --result. Repeatable.
--summary FILERun summary file (summary.json). Repeatable.
--log-root DIRLog root holding LOG_STAMP run directories to scan.
--last NHow many newest run directories under --log-root to read (1 to 65535; default 10).
--target NAMEKeep observations from this target only; exit 2 when no input contains it.
--existing FILESeed weights TSV (MODULE<TAB>SECONDS) to merge with new observations.
--min-samples NUpdate a module only after N selected observations, from 1 to 65535; 1 when absent. Under-sampled existing weights remain unchanged.
--mode MODECombine durations with max (default), latest or mean.
--out FILEAtomically replace FILE with the complete table; - or absent prints to stdout. FILE must differ from every result, summary and existing weights file.

An existing weights file that sets one module more than once is refused with both line numbers.

With no input option, EB_PIPELINE_RESULT supplies one result file. EB_PIPELINE_TARGET supplies --target. A command-line option wins.

Exit 0 means the table was written. Exit 2 is a missing or invalid input.

eb-junit

eb-junit reads one or more result files and writes a JUnit XML report, one suite per file and one case per recorded line. The JUnit report lists the elements. run does not write this report.

eb-junit [--result FILE]... [--reframe-report FILE]... [--out FILE]
         [--property NAME=VALUE]... [--fail-on-failure]
OptionMeaning
--result FILEA result file written by eb-pipeline run --result. Repeat for each file.
--reframe-report FILEA ReFrame report JSON written by reframe --report-file. Repeat for each file.
--out FILEAtomically replace FILE with the complete report. - or absent prints it on stdout. FILE must differ from every result and ReFrame report file.
--property NAME=VALUEAdd the property to every suite. Repeat for more properties; the last value for a name wins.
--fail-on-failureWrite the complete report, then exit 1 when it contains at least one failure.

With no input option, EB_PIPELINE_RESULT supplies one result file. A command-line input wins. Otherwise at least one of --result or --reframe-report is required.

Exit 0 means the report was written, including when a run’s own exit code is not 0. With --fail-on-failure, exit 1 means the report was written and contains a failure. Exit 2 is a missing file, a file that is not a result of schema version 1, or a report that could not be written.

eb-test-map

eb-test-map maps a build list, easystack, or eb-pipeline run --result file to test names, ReFrame selectors, or EESSI module arguments (--format eessi-modules). See The test map for all options and exit statuses.

eb-cvmfs

eb-cvmfs brackets one cache synchronization command in a CVMFS transaction, with optional archive-manifest checks, verification, timeouts, tagging, integrity checks, and Prometheus metrics. See The eb-cvmfs command for all options and exit statuses.

See also