From 3025cf6c4c39092bd739fa519681ff3fd356d68b Mon Sep 17 00:00:00 2001 From: Rohit Goswami Date: Thu, 19 Nov 2026 13:48:36 +0100 Subject: [PATCH 1/3] feat(env): --json prints the change the env and dev lines make eb-local runs the lines in a bash child and prints how its environment then differs from the caller's, as set and unset lists in one line of JSON. A shell that cannot evaluate POSIX text applies that; the reference has a nushell wrapper. Shell functions such as Lmod's module are not variables and do not cross over. --- changelog.d/+env-json.feature.md | 1 + docs/orgmode/reference/cli.org | 16 ++++++ docs/reference/cli.md | 16 ++++++ src/main.zig | 90 +++++++++++++++++++++++++++++++- tests/test_dev_cli_modes.py | 12 +++++ tests/test_env_cli_modes.py | 33 ++++++++++++ 6 files changed, 166 insertions(+), 2 deletions(-) create mode 100644 changelog.d/+env-json.feature.md diff --git a/changelog.d/+env-json.feature.md b/changelog.d/+env-json.feature.md new file mode 100644 index 0000000..04cfbdc --- /dev/null +++ b/changelog.d/+env-json.feature.md @@ -0,0 +1 @@ +`eb-local env --json` and `eb-local dev --json` print the change their lines make to the environment (`set` and `unset`, as one line of JSON) in place of the lines, so a shell that cannot evaluate POSIX text, such as nushell, can apply it. The reference has a nushell wrapper. diff --git a/docs/orgmode/reference/cli.org b/docs/orgmode/reference/cli.org index bcf885e..f3ef1c5 100644 --- a/docs/orgmode/reference/cli.org +++ b/docs/orgmode/reference/cli.org @@ -293,6 +293,21 @@ When stdout is a terminal, it also says on stderr that the lines change nothing | Option | Meaning | |------------+----------------------------------------------------------------------------------------------------------------------| | =--strict= | Check that the EESSI init file exists in the emitted lines and exit 2 if absent (also when =EB_LOCAL_ENV_STRICT=1=). | +| =--json= | Print, in place of the lines, the change they make to the environment, as one line of JSON. | + +With =--json=, eb-local runs the lines in a bash child and prints how that shell's environment then differs from the caller's: ={"schema_version":1,"kind":"env","set":[{"name":...,"value":...}],"unset":[...]}=, both lists sorted by name. A shell that cannot evaluate POSIX lines applies it; in nushell: + +#+begin_src nu +def --env --wrapped eb [...args] { + let change = (^eb-local ...$args --json | from json) + $change.set | reduce --fold {} {|var, acc| $acc | upsert $var.name $var.value } | load-env + hide-env --ignore-errors ...$change.unset +} +eb env +eb dev FFTW/3.3.10-GCC-15.2.0 +#+end_src + +Only variables are in the change. Shell functions and aliases, such as Lmod's =module=, stay in the bash child, so a later =module load= in the other shell is not possible this way; name the modules on the =dev= line. What the lines print goes to stderr. If the lines fail in bash, nothing is printed and the status is 1. ** dev @@ -301,6 +316,7 @@ Print the lines =env= prints, then the lines that load the modules =dev.modules= | Option | Meaning | |---------------------+--------------------------------------------------------------------------------------------| | =--strict= | Check that the EESSI init file exists in the emitted lines and exit 2 if absent (also when =EB_LOCAL_ENV_STRICT=1=). | +| =--json= | Print the change the lines make to the environment as one line of JSON, as =env --json= does, with =kind= =dev=. | | =--buildenv MODULE= | Load MODULE as the build environment whatever the compilers, as =dev.buildenv= does. | A =MODULE= starts with a letter or digit and holds letters, digits, =.=, =_=, =+=, =-= and single slashes, with none at the end. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 45a693e..cc8056c 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -295,6 +295,21 @@ When stdout is a terminal, it also says on stderr that the lines change nothing | Option | Meaning | |---|---| | `--strict` | Check that the EESSI init file exists in the emitted lines and exit 2 if absent (also when `EB_LOCAL_ENV_STRICT=1`). | +| `--json` | Print, in place of the lines, the change they make to the environment, as one line of JSON. | + +With `--json`, eb-local runs the lines in a bash child and prints how that shell's environment then differs from the caller's: `{"schema_version":1,"kind":"env","set":[{"name":...,"value":...}],"unset":[...]}`, both lists sorted by name. A shell that cannot evaluate POSIX lines applies it; in nushell: + +```nu +def --env --wrapped eb [...args] { + let change = (^eb-local ...$args --json | from json) + $change.set | reduce --fold {} {|var, acc| $acc | upsert $var.name $var.value } | load-env + hide-env --ignore-errors ...$change.unset +} +eb env +eb dev FFTW/3.3.10-GCC-15.2.0 +``` + +Only variables are in the change. Shell functions and aliases, such as Lmod's `module`, stay in the bash child, so a later `module load` in the other shell is not possible this way; name the modules on the `dev` line. What the lines print goes to stderr. If the lines fail in bash, nothing is printed and the status is 1. ### dev @@ -303,6 +318,7 @@ Print the lines `env` prints, then the lines that load the modules `dev.modules` | Option | Meaning | |---|---| | `--strict` | Check that the EESSI init file exists in the emitted lines and exit 2 if absent (also when `EB_LOCAL_ENV_STRICT=1`). | +| `--json` | Print the change the lines make to the environment as one line of JSON, as `env --json` does, with `kind` `dev`. | | `--buildenv MODULE` | Load MODULE as the build environment whatever the compilers, as `dev.buildenv` does. | A `MODULE` starts with a letter or digit and holds letters, digits, `.`, `_`, `+`, `-` and single slashes, with none at the end. diff --git a/src/main.zig b/src/main.zig index 0204cb3..d6b8394 100644 --- a/src/main.zig +++ b/src/main.zig @@ -599,10 +599,91 @@ fn envStrict(c: Ctx, g: Global) bool { return v.len > 0 and !std.mem.eql(u8, v, "0") and !std.mem.eql(u8, v, "false"); } +/// One variable of an `EnvChange`. +const EnvVar = struct { + name: []const u8, + value: []const u8, +}; + +/// `env --json` and `dev --json`: what the lines change in the environment of +/// the shell that evaluates them, for a shell that cannot evaluate them. +/// Shell functions and aliases (Lmod's `module`) are not variables and are +/// not in it. +const EnvChange = struct { + schema_version: u32 = jsonout.schema_version, + kind: []const u8, + /// Variables the lines add or change, by name. + set: []const EnvVar, + /// Variables the lines remove, by name. + unset: []const []const u8, +}; + +/// Variables bash keeps for itself; they say nothing about the lines. +const shell_own = [_][]const u8{ "_", "SHLVL", "PWD", "OLDPWD", "BASH_EXECUTION_STRING" }; + +fn shellOwn(name: []const u8) bool { + if (std.mem.startsWith(u8, name, "BASH_FUNC_")) return true; + for (shell_own) |own| if (std.mem.eql(u8, name, own)) return true; + return false; +} + +fn envVarLess(_: void, x: EnvVar, y: EnvVar) bool { + return std.mem.lessThan(u8, x.name, y.name); +} + +/// Run `lines` in a bash child and print, as one JSON document, how its +/// environment then differs from this process's. The child's own output goes +/// to stderr, so stdout carries the document alone. +fn emitEnvChange(c: Ctx, kind: []const u8, lines: []const u8) u8 { + const a = c.a; + const bash = (proc.which(a, "bash", c.env.get("PATH") orelse "/usr/bin:/bin") catch null) orelse "/bin/bash"; + const script = str(a, "exec 3>&1 1>&2\n{s}\nenv -0 >&3\n", .{lines}); + const r = proc.run(a, bash, &.{ bash, "--noprofile", "--norc", "-c", script }, c.envp, .{ + .stdout_limit = 8 << 20, + .stderr_limit = 1 << 20, + .timeout_ms = 300_000, + }) catch |e| { + warn("could not run the {s} lines in bash: {s}", .{ kind, @errorName(e) }); + return 1; + }; + if (r.stderr.len > 0) cli.put(2, r.stderr); + if (r.term.code() != 0) { + warn("the {s} lines failed in bash (status {d}); no change is printed", .{ kind, r.term.code() }); + return 1; + } + var set: std.ArrayList(EnvVar) = .empty; + var after: std.StringHashMapUnmanaged(void) = .empty; + var it = std.mem.splitScalar(u8, r.stdout, 0); + while (it.next()) |entry| { + const eq = std.mem.indexOfScalar(u8, entry, '=') orelse continue; + const name = entry[0..eq]; + const value = entry[eq + 1 ..]; + if (name.len == 0 or shellOwn(name)) continue; + after.put(a, name, {}) catch oom(); + if (c.env.get(name)) |before| { + if (std.mem.eql(u8, before, value)) continue; + } + set.append(a, .{ .name = name, .value = value }) catch oom(); + } + var unset: std.ArrayList([]const u8) = .empty; + var before_it = c.env.iterator(); + while (before_it.next()) |kv| { + const name = kv.key_ptr.*; + if (shellOwn(name) or after.contains(name)) continue; + unset.append(a, name) catch oom(); + } + std.mem.sort(EnvVar, set.items, {}, envVarLess); + std.mem.sort([]const u8, unset.items, {}, lessThanStr); + jsonout.emit(a, EnvChange{ .kind = kind, .set = set.items, .unset = unset.items }); + return 0; +} + fn cmdEnv(c: Ctx, s: config.Settings, g: Global) u8 { const p = place(c, s); + const lines = environmentLines(c, s, p, envStrict(c, g)); + if (g.json_out) return emitEnvChange(c, "env", lines); evalHint(replay(c.a, "env", s, g, p.prefix, &.{})); - cli.put(1, environmentLines(c, s, p, envStrict(c, g))); + cli.put(1, lines); return 0; } @@ -620,7 +701,7 @@ fn cmdDev(c: Ctx, s: config.Settings, g: Global, extra: []const []const u8) u8 { } else mods.append(a, m) catch oom(); }; const installed = place(c, s); - evalHint(replay(a, "dev", s, g, installed.prefix, extra)); + if (!g.json_out) evalHint(replay(a, "dev", s, g, installed.prefix, extra)); var out: std.ArrayList(u8) = .empty; out.appendSlice(a, environmentLines(c, s, installed, envStrict(c, g))) catch oom(); // The modules the program uses first, the build environment last, so @@ -643,6 +724,7 @@ fn cmdDev(c: Ctx, s: config.Settings, g: Global, extra: []const []const u8) u8 { // has picked them. const dev_root = std.fs.path.join(a, &.{ installed.prefix, "dev" }) catch oom(); out.appendSlice(a, str(a, "export EB_LOCAL_DEV_PREFIX={s}\"/$EESSI_VERSION/$EESSI_SOFTWARE_SUBDIR${{EESSI_ACCELERATOR_TARGET:+/$EESSI_ACCELERATOR_TARGET}}\"\n", .{q(a, dev_root)})) catch oom(); + if (g.json_out) return emitEnvChange(c, "dev", out.items); cli.put(1, out.items); if (nonEmpty(c.env.get("LD_LIBRARY_PATH")) != null) { warn("LD_LIBRARY_PATH is set in this shell; programs built here do not need it,", .{}); @@ -4537,6 +4619,8 @@ pub fn main(init: std.process.Init) u8 { .env => { if (std.mem.eql(u8, x, "--strict")) { g.env_strict = true; + } else if (std.mem.eql(u8, x, "--json")) { + g.json_out = true; } else die("env takes only --project, --profile, --config and --strict; got '{s}'", .{x}); }, .profiles => { @@ -4731,6 +4815,8 @@ pub fn main(init: std.process.Init) u8 { .dev => { if (std.mem.eql(u8, x, "--strict")) { g.env_strict = true; + } else if (std.mem.eql(u8, x, "--json")) { + g.json_out = true; } else if (std.mem.eql(u8, x, "--buildenv")) { g.buildenv = value(args, &i, "--buildenv"); } else if (joined(x, "--buildenv")) |v| { diff --git a/tests/test_dev_cli_modes.py b/tests/test_dev_cli_modes.py index 9bedc41..03f39d8 100644 --- a/tests/test_dev_cli_modes.py +++ b/tests/test_dev_cli_modes.py @@ -1,5 +1,6 @@ #!/usr/bin/env python3 """Black-box CLI behavior tests for `eb-local dev` module validation and warnings.""" +import json import os from pathlib import Path import subprocess @@ -45,6 +46,17 @@ def test_invalid_module_names_exit_two(self): self.assertEqual(got.stdout, "") self.assertIn("is not a module name", got.stderr) + def test_json_is_the_change_the_dev_lines_make(self): + got = self.run_dev(["--json"]) + self.assertEqual(got.returncode, 0, got.stderr) + self.assertEqual(got.stdout.count("\n"), 1, got.stdout) + change = json.loads(got.stdout) + self.assertEqual(change["kind"], "dev") + values = {var["name"]: var["value"] for var in change["set"]} + self.assertIn("EB_LOCAL_DEV_PREFIX", values) + self.assertTrue(values["EB_LOCAL_DEV_PREFIX"].startswith(str(self.prefix) + "/dev/"), values) + self.assertNotIn("module load", got.stdout) + def test_names_the_host_ca_bundle_without_replacing_one_the_shell_set(self): got = self.run_dev([]) self.assertEqual(got.returncode, 0, got.stderr) diff --git a/tests/test_env_cli_modes.py b/tests/test_env_cli_modes.py index 835123b..68ccb29 100644 --- a/tests/test_env_cli_modes.py +++ b/tests/test_env_cli_modes.py @@ -1,5 +1,6 @@ #!/usr/bin/env python3 """Black-box CLI behavior tests for `eb-local env` and `--env`.""" +import json from pathlib import Path import subprocess import tempfile @@ -43,6 +44,38 @@ def run_env(self, args, **env_overrides): text=True, ) + def test_json_is_the_change_the_lines_make_to_the_environment(self): + got = self.run_env( + ["env", "--json"], + EESSI_SITE_INSTALL="1", + EASYBUILD_PREFIX="/elsewhere", + KEPT="as it was", + ) + self.assertEqual(got.returncode, 0, got.stderr) + self.assertEqual(got.stdout.count("\n"), 1, got.stdout) + change = json.loads(got.stdout) + self.assertEqual(change["schema_version"], 1) + self.assertEqual(change["kind"], "env") + names = [var["name"] for var in change["set"]] + self.assertEqual(names, sorted(names)) + values = {var["name"]: var["value"] for var in change["set"]} + # what the lines export, and nothing the caller already had with that value + self.assertEqual(values["EESSI_USER_INSTALL"], str(self.prefix)) + self.assertEqual(values["EESSI_CUSTOM_SITE_OPT"], "alpha") + self.assertNotIn("KEPT", values) + self.assertNotIn("HOME", values) + # what the lines unset + self.assertIn("EESSI_SITE_INSTALL", change["unset"]) + self.assertIn("EASYBUILD_PREFIX", change["unset"]) + self.assertNotIn("KEPT", change["unset"]) + self.assertEqual(change["unset"], sorted(change["unset"])) + # bash's own bookkeeping and exported functions are not part of the change + for name in list(values) + change["unset"]: + self.assertNotIn(name, ("_", "SHLVL", "PWD", "OLDPWD")) + self.assertFalse(name.startswith("BASH_FUNC_"), name) + # the lines themselves are not printed + self.assertNotIn("module load", got.stdout) + def test_global_env_flag_matches_env_subcommand(self): sub = self.run_env(["env"]) flag = self.run_env(["--env"]) From fb8d4a2acfd13dd77720f0951c59c4f9c8f92ad2 Mon Sep 17 00:00:00 2001 From: Rohit Goswami Date: Thu, 19 Nov 2026 14:03:45 +0100 Subject: [PATCH 2/3] docs(usage): name --json for env and dev --- docs/orgmode/reference/cli.org | 11 ++++++----- docs/reference/cli.md | 11 ++++++----- src/main.zig | 7 ++++--- 3 files changed, 16 insertions(+), 13 deletions(-) diff --git a/docs/orgmode/reference/cli.org b/docs/orgmode/reference/cli.org index f3ef1c5..693090f 100644 --- a/docs/orgmode/reference/cli.org +++ b/docs/orgmode/reference/cli.org @@ -11,8 +11,8 @@ eb-local [OPTION]... [eb options] eb-local [OPTION]... --explain [eb options] eb-local eb [--] [eb options] eb-local --explain eb [--] [eb options] -eb-local [OPTION]... env [--strict] -eb-local [OPTION]... dev [--strict] [--buildenv MODULE] [MODULE...] +eb-local [OPTION]... env [--strict] [--json] +eb-local [OPTION]... dev [--strict] [--json] [--buildenv MODULE] [MODULE...] eb-local rpath-wrappers [--for FAMILY | --unwrapped] eb-local check [--recursive] [--quiet] [--json] FILE... eb-local check (--module NAME/VERSION... | --dev | --prefix DIR) [--recursive] [--quiet] [--json] @@ -57,15 +57,16 @@ directory you own. tree that variable selects eb-local --explain eb [--] [eb options] the script eb runs, with its arguments; runs nothing - eb-local env [--strict] lines that make your installed modules visible; + eb-local env [--strict] [--json] lines that make your installed modules visible; --strict exits when the EESSI init file is missing (also when EB_LOCAL_ENV_STRICT=1) - eb-local dev [--strict] [--buildenv MODULE] [MODULE...] + eb-local dev [--strict] [--json] [--buildenv MODULE] [MODULE...] the same, plus MODULEs and a build environment, so gcc, make, cmake and meson just work; --strict exits when the EESSI init file is missing (also when EB_LOCAL_ENV_STRICT=1); - --buildenv names the build environment to load + --buildenv names the build environment to load; + --json prints the change as JSON, for env too eb-local rpath-wrappers [--for FAMILY | --unwrapped] lines that put RPATH wrappers for the compilers on PATH first; dev uses them when no build diff --git a/docs/reference/cli.md b/docs/reference/cli.md index cc8056c..7bbc2c5 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -13,8 +13,8 @@ eb-local [OPTION]... [eb options] eb-local [OPTION]... --explain [eb options] eb-local eb [--] [eb options] eb-local --explain eb [--] [eb options] -eb-local [OPTION]... env [--strict] -eb-local [OPTION]... dev [--strict] [--buildenv MODULE] [MODULE...] +eb-local [OPTION]... env [--strict] [--json] +eb-local [OPTION]... dev [--strict] [--json] [--buildenv MODULE] [MODULE...] eb-local rpath-wrappers [--for FAMILY | --unwrapped] eb-local check [--recursive] [--quiet] [--json] FILE... eb-local check (--module NAME/VERSION... | --dev | --prefix DIR) [--recursive] [--quiet] [--json] @@ -59,15 +59,16 @@ directory you own. tree that variable selects eb-local --explain eb [--] [eb options] the script eb runs, with its arguments; runs nothing - eb-local env [--strict] lines that make your installed modules visible; + eb-local env [--strict] [--json] lines that make your installed modules visible; --strict exits when the EESSI init file is missing (also when EB_LOCAL_ENV_STRICT=1) - eb-local dev [--strict] [--buildenv MODULE] [MODULE...] + eb-local dev [--strict] [--json] [--buildenv MODULE] [MODULE...] the same, plus MODULEs and a build environment, so gcc, make, cmake and meson just work; --strict exits when the EESSI init file is missing (also when EB_LOCAL_ENV_STRICT=1); - --buildenv names the build environment to load + --buildenv names the build environment to load; + --json prints the change as JSON, for env too eb-local rpath-wrappers [--for FAMILY | --unwrapped] lines that put RPATH wrappers for the compilers on PATH first; dev uses them when no build diff --git a/src/main.zig b/src/main.zig index d6b8394..74f2ac8 100644 --- a/src/main.zig +++ b/src/main.zig @@ -62,15 +62,16 @@ const usage_text = \\ tree that variable selects \\ eb-local --explain eb [--] [eb options] \\ the script eb runs, with its arguments; runs nothing - \\ eb-local env [--strict] lines that make your installed modules visible; + \\ eb-local env [--strict] [--json] lines that make your installed modules visible; \\ --strict exits when the EESSI init file is missing \\ (also when EB_LOCAL_ENV_STRICT=1) - \\ eb-local dev [--strict] [--buildenv MODULE] [MODULE...] + \\ eb-local dev [--strict] [--json] [--buildenv MODULE] [MODULE...] \\ the same, plus MODULEs and a build environment, \\ so gcc, make, cmake and meson just work; \\ --strict exits when the EESSI init file is missing \\ (also when EB_LOCAL_ENV_STRICT=1); - \\ --buildenv names the build environment to load + \\ --buildenv names the build environment to load; + \\ --json prints the change as JSON, for env too \\ eb-local rpath-wrappers [--for FAMILY | --unwrapped] \\ lines that put RPATH wrappers for the compilers \\ on PATH first; dev uses them when no build From 4d8172a84f28ea50b9313d207ffcb14c8ac67ee5 Mon Sep 17 00:00:00 2001 From: Rohit Goswami Date: Thu, 19 Nov 2026 15:25:40 +0100 Subject: [PATCH 3/3] docs(cli): the nushell example is a plain text block Pygments has no nu lexer, and the site build treats the warning as an error. --- docs/orgmode/reference/cli.org | 2 +- docs/reference/cli.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/orgmode/reference/cli.org b/docs/orgmode/reference/cli.org index 693090f..df75f1c 100644 --- a/docs/orgmode/reference/cli.org +++ b/docs/orgmode/reference/cli.org @@ -298,7 +298,7 @@ When stdout is a terminal, it also says on stderr that the lines change nothing With =--json=, eb-local runs the lines in a bash child and prints how that shell's environment then differs from the caller's: ={"schema_version":1,"kind":"env","set":[{"name":...,"value":...}],"unset":[...]}=, both lists sorted by name. A shell that cannot evaluate POSIX lines applies it; in nushell: -#+begin_src nu +#+begin_src text def --env --wrapped eb [...args] { let change = (^eb-local ...$args --json | from json) $change.set | reduce --fold {} {|var, acc| $acc | upsert $var.name $var.value } | load-env diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 7bbc2c5..a8c2eae 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -300,7 +300,7 @@ When stdout is a terminal, it also says on stderr that the lines change nothing With `--json`, eb-local runs the lines in a bash child and prints how that shell's environment then differs from the caller's: `{"schema_version":1,"kind":"env","set":[{"name":...,"value":...}],"unset":[...]}`, both lists sorted by name. A shell that cannot evaluate POSIX lines applies it; in nushell: -```nu +```text def --env --wrapped eb [...args] { let change = (^eb-local ...$args --json | from json) $change.set | reduce --fold {} {|var, acc| $acc | upsert $var.name $var.value } | load-env