From 32674507c789ac5900b7aae7ea70a7aa760aae47 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 13:49:39 +0000 Subject: [PATCH 01/14] test(conformance): a battery that runs the built shell, as a ratchet (gate G0) First slice of the truthful baseline (owner ruling OD-43, stage S0 of the product review). Until now no test started the built `vsh`: every test file calls library functions, which is how `[ ... ]` and wildcard expansion pass their tests and fail in use. - tests/conformance/battery.jsonl: 126 everyday command lines with the standard output and exit status GNU bash 5.2.21 gave for each (five `rm` cases written by hand from POSIX). - tests/conformance_battery.rs: starts the built shell on each case in a fresh directory and compares output and exit status. Writes a report to target/tmp/conformance-report.txt. - tests/conformance/known_failing.txt: the 67 cases that do not match today. The list is a ratchet: an unlisted failure fails the test, and so does a listed case that passes. Both directions were checked by hand. Starting position on main's code: 59 of 126 match. The shell's own receipt lines on standard output are ignored when comparing; the README records that as an open question, not as settled. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- impl/rust-cli/tests/conformance/README.adoc | 79 +++++ impl/rust-cli/tests/conformance/battery.jsonl | 127 +++++++ .../tests/conformance/known_failing.txt | 78 +++++ impl/rust-cli/tests/conformance_battery.rs | 328 ++++++++++++++++++ 4 files changed, 612 insertions(+) create mode 100644 impl/rust-cli/tests/conformance/README.adoc create mode 100644 impl/rust-cli/tests/conformance/battery.jsonl create mode 100644 impl/rust-cli/tests/conformance/known_failing.txt create mode 100644 impl/rust-cli/tests/conformance_battery.rs diff --git a/impl/rust-cli/tests/conformance/README.adoc b/impl/rust-cli/tests/conformance/README.adoc new file mode 100644 index 0000000..982d25c --- /dev/null +++ b/impl/rust-cli/tests/conformance/README.adoc @@ -0,0 +1,79 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 +// Copyright (c) Jonathan D.A. Jewell += impl/rust-cli/tests/conformance/ — the conformance battery (gate G0) + +Gate G0 of the route plan (owner ruling OD-43): what the project says a front +end can do equals what this battery shows it does. + +== Files + +[cols="2,5"] +|=== +| File | Role + +| `battery.jsonl` +| One case per line: an id, a command line (`cmd`), and the standard output + (`out`) and exit status (`rc`) a reference shell gave for it. The first line + is a comment object. + +| `known_failing.txt` +| The ids of the cases that do not match today. A ratchet: it may only shrink. + +| `../conformance_battery.rs` +| The test. It starts the built `vsh` on each case in a fresh directory and + compares. +|=== + +== Running it + +[source,bash] +---- +cd impl/rust-cli +cargo test --test conformance_battery +---- + +A report of every case is written to `target/tmp/conformance-report.txt`. + +== The ratchet + +* A case that is not on the list and fails is a regression. The test fails. +* A case that is on the list and passes must be taken off it. The test fails + until it is. +* Do not add a case to the list to make a failing build pass. Fix the shell. + If an expectation is wrong, correct the case and say why in the commit. + +== What is compared + +* Standard output, after colour codes and the shell's own `[op:…]` and + `[txn:…]` receipt lines are removed. +* The exit status. +* Not standard error, and not the files left behind. + +The receipt lines are ignored on purpose, and that hides a real difference: a +tracked command such as `mkdir d` prints a receipt on standard output, where +other shells print nothing. A script that captures a command's output would +see it. Whether receipts belong on standard output is an open question for the +engine's command-line tool (stage S1). + +== Where the expectations come from + +Recorded from GNU bash 5.2.21 on 2026-11-17, each case in an empty directory. +The five `rm-NN` cases were written by hand from the POSIX description of `rm`. + +Cases were chosen to give the same result on any Linux system: they create the +files they use and do not depend on what a system directory holds. Of the 126 +cases, 21 use syntax only bash has; `dash` matches bash on the rest. + +== Adding a case + +. Append a line to `battery.jsonl` with a new id of the form `group-NN`. +. Take `out` and `rc` from a reference shell run in an empty directory. +. Run the test. If the case fails, the shell has a gap: add the id to + `known_failing.txt` in the same change and say so. + +== Limits + +* The battery is small and home-made. A public corpus should become the + yardstick for any work on the shell language (review, section 11). +* It runs only the non-interactive path (`vsh -c`). +* A case that does not finish within five seconds counts as failed. diff --git a/impl/rust-cli/tests/conformance/battery.jsonl b/impl/rust-cli/tests/conformance/battery.jsonl new file mode 100644 index 0000000..05dffb9 --- /dev/null +++ b/impl/rust-cli/tests/conformance/battery.jsonl @@ -0,0 +1,127 @@ +{"_comment": "SPDX-License-Identifier: MPL-2.0. Copyright (c) Jonathan D.A. Jewell . One case per line: id, expected exit status (rc), expected standard output (out), command line (cmd). Expectations were recorded from GNU bash 5.2.21 on 2026-11-17, except the rm-NN cases, which were written by hand from the POSIX description of rm. See README.adoc in this directory."} +{"id": "basic-01", "rc": 0, "out": "hello\n", "cmd": "echo hello"} +{"id": "basic-02", "rc": 0, "out": "a b c d\n", "cmd": "echo \"a b\" 'c d'"} +{"id": "basic-03", "rc": 0, "out": "one\ntwo\n", "cmd": "echo one; echo two"} +{"id": "vars-01", "rc": 0, "out": "5\n", "cmd": "x=5; echo $x"} +{"id": "vars-02", "rc": 0, "out": "helloworld\n", "cmd": "x=hello; echo \"${x}world\""} +{"id": "vars-03", "rc": 0, "out": "2\n", "cmd": "x=\"a b\"; echo $x | wc -w"} +{"id": "vars-04", "rc": 0, "out": "default\n", "cmd": "echo ${UNSET:-default}"} +{"id": "vars-05", "rc": 0, "out": "def abc 6\n", "cmd": "x=abcdef; echo ${x#abc} ${x%def} ${#x}"} +{"id": "vars-06", "rc": 0, "out": "HELLO ell\n", "cmd": "x=hello; echo ${x^^} ${x:1:3}"} +{"id": "vars-07", "rc": 0, "out": "bar\n", "cmd": "export FOO=bar; sh -c 'echo $FOO'"} +{"id": "vars-08", "rc": 0, "out": "tmp\n", "cmd": "FOO=tmp sh -c 'echo $FOO'"} +{"id": "vars-09", "rc": 0, "out": "1\n", "cmd": "echo $HOME | grep -c /"} +{"id": "vars-10", "rc": 0, "out": "0\n1\n", "cmd": "echo $? ; false; echo $?"} +{"id": "vars-11", "rc": 0, "out": "1\n", "cmd": "echo \"$$\" | grep -c '[0-9]'"} +{"id": "arith-01", "rc": 0, "out": "7\n", "cmd": "echo $((1+2*3))"} +{"id": "arith-02", "rc": 0, "out": "3 49\n", "cmd": "x=7; echo $((x % 4)) $((x ** 2))"} +{"id": "arith-03", "rc": 0, "out": "1\n", "cmd": "i=0; i=$((i+1)); echo $i"} +{"id": "arith-04", "rc": 0, "out": "7\n", "cmd": "((x=3+4)); echo $x"} +{"id": "arith-05", "rc": 0, "out": "10\n", "cmd": "let y=2*5; echo $y"} +{"id": "subst-01", "rc": 0, "out": "nested\n", "cmd": "echo $(echo nested)"} +{"id": "subst-02", "rc": 0, "out": "today: 20\n", "cmd": "echo \"today: $(date +%Y | cut -c1-2)\""} +{"id": "subst-03", "rc": 0, "out": "back\n", "cmd": "x=`echo back`; echo $x"} +{"id": "subst-04", "rc": 0, "out": "deep\n", "cmd": "echo $(echo $(echo deep))"} +{"id": "pipe-01", "rc": 0, "out": "HELLO\n", "cmd": "echo hello | tr a-z A-Z"} +{"id": "pipe-02", "rc": 0, "out": "a\nb\n", "cmd": "printf 'b\\na\\nc\\n' | sort | head -2"} +{"id": "pipe-03", "rc": 0, "out": "3\n", "cmd": "echo hi | cat | cat | wc -c"} +{"id": "pipe-04", "rc": 0, "out": "0\n", "cmd": "false | true; echo $?"} +{"id": "pipe-05", "rc": 0, "out": "15\n", "cmd": "seq 1 5 | awk '{s+=$1} END{print s}'"} +{"id": "redir-01", "rc": 0, "out": "x\n", "cmd": "echo x > f; cat f"} +{"id": "redir-02", "rc": 0, "out": "2\n", "cmd": "echo x > f; echo y >> f; wc -l < f"} +{"id": "redir-03", "rc": 0, "out": "rc=2\n", "cmd": "ls /nonexistent 2>/dev/null; echo rc=$?"} +{"id": "redir-04", "rc": 0, "out": "1\n", "cmd": "ls /nonexistent > out 2>&1; grep -c . out"} +{"id": "redir-05", "rc": 0, "out": "herestring\n", "cmd": "cat <<< \"herestring\""} +{"id": "redir-06", "rc": 0, "out": "ok\n", "cmd": "echo err >&2 2>/dev/null; echo ok"} +{"id": "logic-01", "rc": 0, "out": "yes\n", "cmd": "true && echo yes || echo no"} +{"id": "logic-02", "rc": 0, "out": "no\n", "cmd": "false && echo yes || echo no"} +{"id": "logic-03", "rc": 0, "out": "dir\n", "cmd": "[ -d / ] && echo dir"} +{"id": "logic-04", "rc": 0, "out": "ok\n", "cmd": "[ 3 -gt 2 ] && [ a = a ] && echo ok"} +{"id": "logic-05", "rc": 0, "out": "match\n", "cmd": "[[ abc == a* ]] && echo match"} +{"id": "logic-06", "rc": 0, "out": "file\n", "cmd": "test -f /etc/passwd && echo file"} +{"id": "logic-07", "rc": 0, "out": "negated\n", "cmd": "! false && echo negated"} +{"id": "if-01", "rc": 0, "out": "t\n", "cmd": "if true; then echo t; else echo f; fi"} +{"id": "if-02", "rc": 0, "out": "b\n", "cmd": "if [ 1 -eq 2 ]; then echo a; elif [ 1 -eq 1 ]; then echo b; fi"} +{"id": "if-03", "rc": 0, "out": "big\n", "cmd": "x=3; if (( x > 2 )); then echo big; fi"} +{"id": "loop-01", "rc": 0, "out": "1\n2\n3\n", "cmd": "for i in 1 2 3; do echo $i; done"} +{"id": "loop-02", "rc": 0, "out": "n1\nn2\nn3\n", "cmd": "for i in $(seq 3); do echo n$i; done"} +{"id": "loop-03", "rc": 0, "out": "p1\np2\n", "cmd": "mkdir zzdir; touch zzdir/p1 zzdir/p2; for f in zzdir/p*; do echo ${f##*/}; done"} +{"id": "loop-04", "rc": 0, "out": "3\n", "cmd": "i=0; while [ $i -lt 3 ]; do i=$((i+1)); done; echo $i"} +{"id": "loop-05", "rc": 0, "out": "0\n1\n2\n", "cmd": "for ((i=0;i<3;i++)); do echo $i; done"} +{"id": "loop-06", "rc": 0, "out": "1\n3\n", "cmd": "for i in 1 2 3; do [ $i = 2 ] && continue; echo $i; done"} +{"id": "loop-07", "rc": 0, "out": "1\n", "cmd": "for i in 1 2 3; do [ $i = 2 ] && break; echo $i; done"} +{"id": "loop-08", "rc": 0, "out": "once\n", "cmd": "until false; do echo once; break; done"} +{"id": "loop-09", "rc": 0, "out": "got line\n", "cmd": "while read l; do echo \"got $l\"; done <<< \"line\""} +{"id": "case-01", "rc": 0, "out": "starts\n", "cmd": "case foo in f*) echo starts;; *) echo other;; esac"} +{"id": "case-02", "rc": 0, "out": "ab\n", "cmd": "x=b; case $x in a|b) echo ab;; esac"} +{"id": "func-01", "rc": 0, "out": "in f arg\n", "cmd": "f() { echo \"in f $1\"; }; f arg"} +{"id": "func-02", "rc": 0, "out": "3\n", "cmd": "f() { return 3; }; f; echo $?"} +{"id": "func-03", "rc": 0, "out": "inner\nouter\n", "cmd": "f() { local v=inner; echo $v; }; v=outer; f; echo $v"} +{"id": "func-04", "rc": 0, "out": "g\n", "cmd": "function g { echo g; }; g"} +{"id": "func-05", "rc": 0, "out": "3\n", "cmd": "f() { echo $#; }; f a b c"} +{"id": "func-06", "rc": 0, "out": "120\n", "cmd": "fact() { [ $1 -le 1 ] && echo 1 || echo $(( $1 * $(fact $(($1-1))) )); }; fact 5"} +{"id": "glob-01", "rc": 0, "out": "a.txt b.txt\n", "cmd": "touch a.txt b.txt c.md; echo *.txt"} +{"id": "glob-02", "rc": 0, "out": "a1 a2\n", "cmd": "touch a1 a2 b1; echo a?"} +{"id": "glob-03", "rc": 0, "out": "*.nomatch\n", "cmd": "touch x.rs; echo *.nomatch"} +{"id": "glob-04", "rc": 0, "out": "d/e/f\n", "cmd": "mkdir -p d/e; touch d/e/f; echo d/*/f"} +{"id": "glob-05", "rc": 0, "out": "a1 a2 b1 b2\n", "cmd": "echo {a,b}{1,2}"} +{"id": "glob-06", "rc": 0, "out": "1 2 3 4 5\n", "cmd": "echo {1..5}"} +{"id": "quote-01", "rc": 0, "out": "single $x double $x\n", "cmd": "echo 'single $x' \"double \\$x\""} +{"id": "quote-02", "rc": 0, "out": "a\\tb a\\tb\n", "cmd": "echo \"a\\tb\" 'a\\tb'"} +{"id": "quote-03", "rc": 0, "out": "*\n", "cmd": "x='*'; echo \"$x\""} +{"id": "quote-04", "rc": 0, "out": "tab\there\n", "cmd": "echo $'tab\\there'"} +{"id": "array-01", "rc": 0, "out": "y 3\n", "cmd": "a=(x y z); echo ${a[1]} ${#a[@]}"} +{"id": "array-02", "rc": 0, "out": "1\n2\n3\n", "cmd": "a=(1 2 3); for v in \"${a[@]}\"; do echo $v; done"} +{"id": "array-03", "rc": 0, "out": "v\n", "cmd": "declare -A m; m[k]=v; echo ${m[k]}"} +{"id": "sub-01", "rc": 1, "out": "1\n0\n", "cmd": "mkdir zzdir; (cd zzdir; pwd | grep -c zzdir); pwd | grep -c zzdir"} +{"id": "sub-02", "rc": 0, "out": "1\n", "cmd": "x=1; (x=2); echo $x"} +{"id": "sub-03", "rc": 0, "out": "2\n", "cmd": "{ echo a; echo b; } | wc -l"} +{"id": "sub-04", "rc": 0, "out": "a\ndone\n", "cmd": "echo a & wait; echo done"} +{"id": "builtin-01", "rc": 0, "out": "1\n", "cmd": "mkdir zzdir; cd zzdir && pwd | grep -c zzdir"} +{"id": "builtin-02", "rc": 0, "out": "abc-42\n", "cmd": "printf '%s-%d\\n' abc 42"} +{"id": "builtin-03", "rc": 0, "out": "two\n", "cmd": "read a b <<< \"one two\"; echo $b"} +{"id": "builtin-04", "rc": 0, "out": "q 3\n", "cmd": "set -- p q r; echo $2 $#"} +{"id": "builtin-05", "rc": 1, "out": "", "cmd": "set -e; false; echo notreached"} +{"id": "builtin-06", "rc": 0, "out": "[]\n", "cmd": "x=1; unset x; echo \"[${x}]\""} +{"id": "builtin-07", "rc": 0, "out": "1\n", "cmd": "type echo | head -1 | grep -c echo"} +{"id": "builtin-08", "rc": 127, "out": "", "cmd": "alias ll='echo aliased'; ll"} +{"id": "builtin-09", "rc": 0, "out": "hi\nbye\n", "cmd": "trap 'echo bye' EXIT; echo hi"} +{"id": "builtin-10", "rc": 0, "out": "evaled\n", "cmd": "eval 'echo evaled'"} +{"id": "builtin-11", "rc": 0, "out": "ok\n", "cmd": "shift 2>/dev/null; echo ok"} +{"id": "builtin-12", "rc": 0, "out": "1\n", "cmd": "mkdir zzdir; pushd zzdir >/dev/null; pwd | grep -c zzdir; popd >/dev/null"} +{"id": "builtin-13", "rc": 0, "out": "nonl\n", "cmd": "echo -n nonl; echo"} +{"id": "builtin-14", "rc": 0, "out": "2\n", "cmd": "echo -e 'a\\nb' | wc -l"} +{"id": "builtin-15", "rc": 7, "out": "", "cmd": "exit 7"} +{"id": "builtin-16", "rc": 0, "out": "1\n", "cmd": "command -v ls | grep -c ls"} +{"id": "builtin-17", "rc": 0, "out": "0\n", "cmd": "true; echo ${PIPESTATUS[0]}"} +{"id": "fs-01", "rc": 0, "out": "c\n", "cmd": "mkdir -p a/b/c && ls a/b"} +{"id": "fs-02", "rc": 0, "out": "gone\n", "cmd": "mkdir d && rmdir d && echo gone"} +{"id": "fs-03", "rc": 0, "out": "data\n", "cmd": "echo data > f; cp f g; cat g"} +{"id": "fs-04", "rc": 0, "out": "data\n", "cmd": "echo data > f; mv f g; cat g"} +{"id": "fs-05", "rc": 0, "out": "600\n", "cmd": "touch f; chmod 600 f; stat -c %a f"} +{"id": "fs-06", "rc": 0, "out": "target\n", "cmd": "ln -s target link; readlink link"} +{"id": "fs-07", "rc": 0, "out": "1\n", "cmd": "touch f; ls -la f | wc -l"} +{"id": "fs-08", "rc": 0, "out": "2\n", "cmd": "echo a > f; cat f f | wc -l"} +{"id": "ext-01", "rc": 0, "out": "1\n", "cmd": "ls / | head -1 | wc -l"} +{"id": "ext-02", "rc": 0, "out": "1\n", "cmd": "env | grep -c '^PATH='"} +{"id": "ext-03", "rc": 0, "out": "4\n", "cmd": "sh -c 'exit 4'; echo $?"} +{"id": "ext-04", "rc": 0, "out": "3\n", "cmd": "printf abc | wc -c"} +{"id": "ext-05", "rc": 0, "out": "slept\n", "cmd": "sleep 0.01; echo slept"} +{"id": "ext-06", "rc": 0, "out": "aXc\n", "cmd": "echo abc | sed 's/b/X/'"} +{"id": "ext-07", "rc": 0, "out": "1\n", "cmd": "find . -maxdepth 0 | wc -l"} +{"id": "ext-08", "rc": 0, "out": "2\n", "cmd": "X=1 Y=2 env | grep -c '^[XY]='"} +{"id": "redir-07", "rc": 0, "out": "heredoc 2\n", "cmd": "cat < a.txt; echo two > b.txt; cat *.txt"} +{"id": "loop-10", "rc": 0, "out": "a1\na2\nb1\nb2\n", "cmd": "for i in a b; do for j in 1 2; do echo $i$j; done; done"} +{"id": "subst-05", "rc": 0, "out": "hi\n", "cmd": "x=$(echo hi); echo $x"} +{"id": "builtin-18", "rc": 3, "out": "", "cmd": "true; exit 3"} +{"id": "builtin-19", "rc": 1, "out": "", "cmd": "false; exit"} +{"id": "fs-09", "rc": 0, "out": "3\n", "cmd": "touch a b c; ls | wc -l"} +{"id": "fs-10", "rc": 0, "out": "2\n", "cmd": "mkdir x y; ls | wc -l"} +{"id": "rm-01", "rc": 0, "out": "0\n", "cmd": "touch f; rm f; ls | wc -l"} +{"id": "rm-02", "rc": 0, "out": "rc=1\n", "cmd": "rm nosuch; echo rc=$?"} +{"id": "rm-03", "rc": 0, "out": "rc=0\n", "cmd": "rm -f nosuch; echo rc=$?"} +{"id": "rm-04", "rc": 0, "out": "0\n", "cmd": "mkdir d; touch d/x; rm -r d; ls | wc -l"} +{"id": "rm-05", "rc": 0, "out": "0\n", "cmd": "touch a b; rm a b; ls | wc -l"} diff --git a/impl/rust-cli/tests/conformance/known_failing.txt b/impl/rust-cli/tests/conformance/known_failing.txt new file mode 100644 index 0000000..75a2bf0 --- /dev/null +++ b/impl/rust-cli/tests/conformance/known_failing.txt @@ -0,0 +1,78 @@ +# SPDX-License-Identifier: MPL-2.0 +# Copyright (c) Jonathan D.A. Jewell +# +# Cases of battery.jsonl that do not match the reference today. +# This list is a ratchet: it may only shrink. A case that starts to pass +# must be removed here, and the test fails until it is. Never add a case to +# make a failing build pass; fix the shell or, if the reference expectation +# is wrong, correct the case and say why in the commit. +# +# Format: one case id per line; text after `#` is a note (here, the command). + +vars-05 # x=abcdef; echo ${x#abc} ${x%def} ${#x} +vars-06 # x=hello; echo ${x^^} ${x:1:3} +vars-07 # export FOO=bar; sh -c 'echo $FOO' +vars-08 # FOO=tmp sh -c 'echo $FOO' +vars-11 # echo "$$" | grep -c '[0-9]' +arith-01 # echo $((1+2*3)) +arith-02 # x=7; echo $((x % 4)) $((x ** 2)) +arith-04 # ((x=3+4)); echo $x +arith-05 # let y=2*5; echo $y +subst-01 # echo $(echo nested) +subst-02 # echo "today: $(date +%Y | cut -c1-2)" +subst-03 # x=`echo back`; echo $x +subst-04 # echo $(echo $(echo deep)) +pipe-05 # seq 1 5 | awk '{s+=$1} END{print s}' +redir-03 # ls /nonexistent 2>/dev/null; echo rc=$? +redir-04 # ls /nonexistent > out 2>&1; grep -c . out +logic-03 # [ -d / ] && echo dir +logic-04 # [ 3 -gt 2 ] && [ a = a ] && echo ok +logic-05 # [[ abc == a* ]] && echo match +logic-07 # ! false && echo negated +if-02 # if [ 1 -eq 2 ]; then echo a; elif [ 1 -eq 1 ]; then echo b; fi +if-03 # x=3; if (( x > 2 )); then echo big; fi +loop-02 # for i in $(seq 3); do echo n$i; done +loop-03 # mkdir zzdir; touch zzdir/p1 zzdir/p2; for f in zzdir/p*; do echo ${f##*/}; done +loop-04 # i=0; while [ $i -lt 3 ]; do i=$((i+1)); done; echo $i +loop-05 # for ((i=0;i<3;i++)); do echo $i; done +loop-06 # for i in 1 2 3; do [ $i = 2 ] && continue; echo $i; done +loop-07 # for i in 1 2 3; do [ $i = 2 ] && break; echo $i; done +loop-08 # until false; do echo once; break; done +loop-09 # while read l; do echo "got $l"; done <<< "line" +func-06 # fact() { [ $1 -le 1 ] && echo 1 || echo $(( $1 * $(fact $(($1-1))) )); }; fact 5 +glob-01 # touch a.txt b.txt c.md; echo *.txt +glob-02 # touch a1 a2 b1; echo a? +glob-04 # mkdir -p d/e; touch d/e/f; echo d/*/f +glob-05 # echo {a,b}{1,2} +glob-06 # echo {1..5} +quote-03 # x='*'; echo "$x" +quote-04 # echo $'tab\there' +array-01 # a=(x y z); echo ${a[1]} ${#a[@]} +array-02 # a=(1 2 3); for v in "${a[@]}"; do echo $v; done +array-03 # declare -A m; m[k]=v; echo ${m[k]} +sub-01 # mkdir zzdir; (cd zzdir; pwd | grep -c zzdir); pwd | grep -c zzdir +sub-03 # { echo a; echo b; } | wc -l +sub-04 # echo a & wait; echo done +builtin-03 # read a b <<< "one two"; echo $b +builtin-05 # set -e; false; echo notreached +builtin-06 # x=1; unset x; echo "[${x}]" +builtin-07 # type echo | head -1 | grep -c echo +builtin-12 # mkdir zzdir; pushd zzdir >/dev/null; pwd | grep -c zzdir; popd >/dev/null +builtin-15 # exit 7 +builtin-16 # command -v ls | grep -c ls +builtin-17 # true; echo ${PIPESTATUS[0]} +fs-01 # mkdir -p a/b/c && ls a/b +ext-08 # X=1 Y=2 env | grep -c '^[XY]=' +redir-07 # cat < a.txt; echo two > b.txt; cat *.txt +loop-10 # for i in a b; do for j in 1 2; do echo $i$j; done; done +subst-05 # x=$(echo hi); echo $x +builtin-18 # true; exit 3 +builtin-19 # false; exit +fs-09 # touch a b c; ls | wc -l +fs-10 # mkdir x y; ls | wc -l +rm-02 # rm nosuch; echo rc=$? +rm-03 # rm -f nosuch; echo rc=$? +rm-04 # mkdir d; touch d/x; rm -r d; ls | wc -l diff --git a/impl/rust-cli/tests/conformance_battery.rs b/impl/rust-cli/tests/conformance_battery.rs new file mode 100644 index 0000000..fd53965 --- /dev/null +++ b/impl/rust-cli/tests/conformance_battery.rs @@ -0,0 +1,328 @@ +// SPDX-License-Identifier: MPL-2.0 +// Copyright (c) Jonathan D.A. Jewell +//! Gate G0 (use): what the shell is said to do equals what this battery shows. +//! +//! Every case in `tests/conformance/battery.jsonl` is one command line with the +//! standard output and exit status that a reference shell gave for it. This +//! test starts the *built* `vsh` program on each line, in a fresh directory, +//! and compares. +//! +//! `tests/conformance/known_failing.txt` lists the cases that do not match +//! today. The list is a ratchet: +//! +//! * a case that is *not* listed and fails is a regression, and fails the test; +//! * a case that *is* listed and passes must be taken off the list, and fails +//! the test until it is. The list can only shrink. +//! +//! What is compared, and what is not: +//! +//! * standard output, after colour codes and the shell's own `[op:…]` and +//! `[txn:…]` receipt lines are removed; +//! * the exit status; +//! * not standard error, and not the files left behind. +//! +//! A report of every case is written to +//! `$CARGO_TARGET_TMPDIR/conformance-report.txt`. +//! +//! See `tests/conformance/README.adoc` for how to add a case. + +use std::collections::{BTreeMap, BTreeSet}; +use std::io::Read; +use std::process::{Command, Stdio}; +use std::time::{Duration, Instant}; + +const BATTERY: &str = include_str!("conformance/battery.jsonl"); +const KNOWN_FAILING: &str = include_str!("conformance/known_failing.txt"); + +/// Longest a single case may run. A case that does not finish counts as failed. +const TIME_LIMIT: Duration = Duration::from_secs(5); + +/// How many cases run at once. +const WORKERS: usize = 8; + +#[derive(Debug, Clone)] +struct Case { + id: String, + rc: i32, + out: String, + cmd: String, +} + +#[derive(Debug)] +struct Outcome { + id: String, + passed: bool, + detail: String, +} + +fn cases() -> Vec { + BATTERY + .lines() + .filter(|line| !line.trim().is_empty()) + .filter_map(|line| { + let value: serde_json::Value = + serde_json::from_str(line).expect("battery.jsonl: every line is a JSON object"); + // The first line is a comment object carrying the licence header. + let id = value.get("id")?.as_str()?.to_string(); + let field = |name: &str| { + value + .get(name) + .unwrap_or_else(|| panic!("battery.jsonl: case {id} has no `{name}`")) + .clone() + }; + Some(Case { + rc: i32::try_from(field("rc").as_i64().expect("`rc` is a number")) + .expect("`rc` fits in i32"), + out: field("out") + .as_str() + .expect("`out` is a string") + .to_string(), + cmd: field("cmd") + .as_str() + .expect("`cmd` is a string") + .to_string(), + id, + }) + }) + .collect() +} + +fn known_failing() -> BTreeSet { + KNOWN_FAILING + .lines() + .map(|line| line.split('#').next().unwrap_or("").trim()) + .filter(|id| !id.is_empty()) + .map(str::to_string) + .collect() +} + +/// Removes terminal colour codes and the shell's own receipt lines. +fn normalise(raw: &str) -> String { + let mut plain = String::with_capacity(raw.len()); + let mut chars = raw.chars().peekable(); + while let Some(ch) = chars.next() { + if ch == '\u{1b}' && chars.peek() == Some(&'[') { + chars.next(); + // A control sequence ends at the first byte in the range @ to ~. + for inner in chars.by_ref() { + if ('@'..='~').contains(&inner) { + break; + } + } + } else { + plain.push(ch); + } + } + let mut kept = String::with_capacity(plain.len()); + for line in plain.split_inclusive('\n') { + if line.starts_with("[op:") || line.starts_with("[txn:") { + continue; + } + kept.push_str(line); + } + kept +} + +/// Runs one case and says whether output and exit status match. +fn run(case: &Case) -> Outcome { + let dir = tempfile::tempdir().expect("temporary directory"); + let mut child = Command::new(env!("CARGO_BIN_EXE_vsh")) + .arg("-r") + .arg(dir.path()) + .arg("-c") + .arg(&case.cmd) + .current_dir(dir.path()) + .env("NO_COLOR", "1") + .env_remove("RUST_BACKTRACE") + .stdin(Stdio::null()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .spawn() + .expect("start the built shell"); + + let started = Instant::now(); + let status = loop { + match child.try_wait().expect("wait for the shell") { + Some(status) => break Some(status), + None if started.elapsed() >= TIME_LIMIT => { + let _ = child.kill(); + let _ = child.wait(); + break None; + } + None => std::thread::sleep(Duration::from_millis(2)), + } + }; + + let mut raw = Vec::new(); + if let Some(mut stdout) = child.stdout.take() { + let _ = stdout.read_to_end(&mut raw); + } + let got_out = normalise(&String::from_utf8_lossy(&raw)); + + let (passed, detail) = match status { + None => (false, format!("did not finish within {TIME_LIMIT:?}")), + Some(status) => { + let got_rc = status.code().unwrap_or(-1); + if got_out == case.out && got_rc == case.rc { + (true, String::new()) + } else { + ( + false, + format!( + "expected exit {} and {:?}; got exit {} and {:?}", + case.rc, + case.out, + got_rc, + shorten(&got_out) + ), + ) + } + } + }; + Outcome { + id: case.id.clone(), + passed, + detail, + } +} + +fn shorten(text: &str) -> String { + const LIMIT: usize = 120; + if text.chars().count() <= LIMIT { + text.to_string() + } else { + let head: String = text.chars().take(LIMIT).collect(); + format!("{head}…") + } +} + +fn group_of(id: &str) -> &str { + id.rsplit_once('-').map_or(id, |(group, _)| group) +} + +fn run_all(cases: &[Case]) -> Vec { + let chunk = cases.len().div_ceil(WORKERS).max(1); + let mut outcomes: Vec = std::thread::scope(|scope| { + let handles: Vec<_> = cases + .chunks(chunk) + .map(|part| scope.spawn(move || part.iter().map(run).collect::>())) + .collect(); + handles + .into_iter() + .flat_map(|handle| handle.join().expect("worker finished")) + .collect() + }); + let order: BTreeMap<&str, usize> = cases + .iter() + .enumerate() + .map(|(index, case)| (case.id.as_str(), index)) + .collect(); + outcomes.sort_by_key(|outcome| order[outcome.id.as_str()]); + outcomes +} + +fn write_report(cases: &[Case], outcomes: &[Outcome], known: &BTreeSet) -> String { + let mut by_group: BTreeMap<&str, (usize, usize)> = BTreeMap::new(); + for outcome in outcomes { + let entry = by_group.entry(group_of(&outcome.id)).or_default(); + entry.1 += 1; + if outcome.passed { + entry.0 += 1; + } + } + let passed = outcomes.iter().filter(|outcome| outcome.passed).count(); + let mut report = format!( + "conformance battery: {passed} of {} cases match the reference\n\n", + outcomes.len() + ); + for (group, (ok, total)) in &by_group { + report.push_str(&format!("{group:<10} {ok:>3} / {total}\n")); + } + report.push('\n'); + for (case, outcome) in cases.iter().zip(outcomes) { + let mark = match (outcome.passed, known.contains(&outcome.id)) { + (true, false) => "ok ", + (false, true) => "FAIL", + (true, true) => "ok!!", + (false, false) => "NEW!", + }; + report.push_str(&format!( + "{mark} {:<12} {}\n", + outcome.id, + case.cmd.replace('\n', "\\n") + )); + if !outcome.passed { + report.push_str(&format!(" {}\n", outcome.detail)); + } + } + let path = std::path::Path::new(env!("CARGO_TARGET_TMPDIR")).join("conformance-report.txt"); + let _ = std::fs::write(&path, &report); + report +} + +#[test] +fn battery_matches_the_reference_except_for_the_known_failures() { + let cases = cases(); + assert!( + cases.len() >= 100, + "the battery should hold at least 100 cases, found {}", + cases.len() + ); + + let ids: BTreeSet<&str> = cases.iter().map(|case| case.id.as_str()).collect(); + assert_eq!(ids.len(), cases.len(), "case ids must be unique"); + + let known = known_failing(); + let stale: Vec<&String> = known + .iter() + .filter(|id| !ids.contains(id.as_str())) + .collect(); + assert!( + stale.is_empty(), + "known_failing.txt names cases that are not in the battery: {stale:?}" + ); + + let outcomes = run_all(&cases); + let report = write_report(&cases, &outcomes, &known); + + let regressions: Vec = outcomes + .iter() + .filter(|outcome| !outcome.passed && !known.contains(&outcome.id)) + .map(|outcome| format!("{}: {}", outcome.id, outcome.detail)) + .collect(); + let now_passing: Vec<&str> = outcomes + .iter() + .filter(|outcome| outcome.passed && known.contains(&outcome.id)) + .map(|outcome| outcome.id.as_str()) + .collect(); + + assert!( + regressions.is_empty() && now_passing.is_empty(), + "\n{report}\n\ + Regressions (not on the known-failing list, and failing): {regressions:#?}\n\ + Now passing (take these off tests/conformance/known_failing.txt): {now_passing:?}\n" + ); +} + +#[test] +fn receipt_lines_and_colour_codes_are_removed_before_comparing() { + let raw = "\u{1b}[32mgreen\u{1b}[0m\n[op:1f163398] mkdir d\nkept\n[txn:558b5380] begin t\n"; + assert_eq!(normalise(raw), "green\nkept\n"); +} + +#[test] +fn the_known_failing_list_ignores_comments_and_blank_lines() { + let parsed: BTreeSet = "# heading\n\nlogic-03 # bracket test\n \nloop-01\n" + .lines() + .map(|line| line.split('#').next().unwrap_or("").trim()) + .filter(|id| !id.is_empty()) + .map(str::to_string) + .collect(); + assert_eq!( + parsed, + ["logic-03", "loop-01"] + .iter() + .map(|s| s.to_string()) + .collect() + ); +} From 017ba69a01a9e89fe86d7d0b30bce008fbed4f4c Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 13:54:56 +0000 Subject: [PATCH 02/14] fix(shell): bracket test works, exit keeps its status, no fixed wait per program Second slice of the truthful baseline (OD-43). Seven battery cases leave the known-failing list: 66 of 126 now match (was 59). - `[ ... ]` always failed with "missing ']'": the parser checked for and removed the closing bracket, and the evaluator then required it again. The evaluator is now told the bracket is already gone. - `exit N` left with status 0 and skipped the EXIT trap in scripts. `exit` now takes a status (low eight bits, as POSIX has it), keeps the last command's status when given none, refuses a non-number with status 2, and the EXIT trap runs before the shell leaves. The interactive shell leaves with the same status. - Every external program was checked on every 50 ms, so each one took at least that long: 100 runs of a short program took 5.07 s. The check now starts at 0.2 ms and doubles to a ceiling of 20 ms. The same 100 runs take about 0.3 s. Tests: tests/external_wait_tests.rs runs the built shell (the first of its two tests fails when the 50 ms wait is put back: checked). cargo fmt and clippy with -D warnings are clean. cargo test --no-fail-fast: 832 passed, 0 failed, 15 ignored. `[ -d / ]` and `exit 5` were also checked by hand in an interactive session. Not done here: a failing built-in still ends a `-c` script; `break`, `continue` and `! cmd` are still missing. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- impl/rust-cli/src/executable.rs | 25 +++++++-- impl/rust-cli/src/external.rs | 30 +++++++++- impl/rust-cli/src/main.rs | 12 +++- impl/rust-cli/src/parser.rs | 14 ++++- .../tests/conformance/known_failing.txt | 7 --- impl/rust-cli/tests/external_wait_tests.rs | 56 +++++++++++++++++++ 6 files changed, 127 insertions(+), 17 deletions(-) create mode 100644 impl/rust-cli/tests/external_wait_tests.rs diff --git a/impl/rust-cli/src/executable.rs b/impl/rust-cli/src/executable.rs index aeb72a2..ca8841d 100644 --- a/impl/rust-cli/src/executable.rs +++ b/impl/rust-cli/src/executable.rs @@ -279,7 +279,22 @@ impl ExecutableCommand for Command { } // Exit commands - Command::Exit | Command::Quit => Ok(ExecutionResult::Exit), + Command::Exit { code } => { + // `exit` alone keeps the status of the last command. Only the low + // eight bits of a given status reach the caller (POSIX). + if let Some(code) = code { + let text = crate::parser::expand_variables(code, state); + match text.trim().parse::() { + Ok(status) => state.last_exit_code = (status & 0xFF) as i32, + Err(_) => { + eprintln!("vsh: exit: {}: numeric argument required", text); + state.last_exit_code = 2; + } + } + } + Ok(ExecutionResult::Exit) + } + Command::Quit => Ok(ExecutionResult::Exit), // Navigation (built-ins but not reversible) Command::Ls { path, redirects } => { @@ -442,12 +457,14 @@ impl ExecutableCommand for Command { .map(|arg| crate::parser::expand_variables(arg, state)) .collect(); + // The parser has already checked for the closing `]` and + // removed it, so the arguments are a plain `test` expression. let exit_code = if redirects.is_empty() { - crate::test_command::execute_test(&expanded_args, true)? + crate::test_command::execute_test(&expanded_args, false)? } else { let mut code = 0; redirection::capture_and_redirect(redirects, state, |_s| { - code = crate::test_command::execute_test(&expanded_args, true)?; + code = crate::test_command::execute_test(&expanded_args, false)?; Ok(()) })?; code @@ -1244,7 +1261,7 @@ impl ExecutableCommand for Command { format!("history {}", count) } } - Command::Exit => "exit".to_string(), + Command::Exit { .. } => "exit".to_string(), Command::Quit => "quit".to_string(), Command::Begin { name } => format!("begin {}", name), Command::Commit => "commit".to_string(), diff --git a/impl/rust-cli/src/external.rs b/impl/rust-cli/src/external.rs index 4dbc9f9..2450485 100644 --- a/impl/rust-cli/src/external.rs +++ b/impl/rust-cli/src/external.rs @@ -16,6 +16,27 @@ use crate::redirection::{RedirectSetup, Redirection}; use crate::signals; use crate::state::ShellState; +/// The wait between two checks on a running child. +/// +/// It starts short, so a program that finishes at once is not held up, and +/// doubles up to a ceiling, so a long-running one costs almost nothing to +/// watch. The ceiling is also the longest an interrupt waits to be noticed. +struct Pause(Duration); + +impl Pause { + const FIRST: Duration = Duration::from_micros(200); + const LONGEST: Duration = Duration::from_millis(20); + + fn new() -> Self { + Self(Self::FIRST) + } + + fn sleep(&mut self) { + std::thread::sleep(self.0); + self.0 = (self.0 * 2).min(Self::LONGEST); + } +} + /// Execute an external command with I/O redirections /// /// Handles: @@ -64,6 +85,7 @@ pub fn execute_external_with_redirects( .context(format!("Failed to spawn: {}", program))?; // Poll for exit while checking for interrupt + let mut pause = Pause::new(); loop { match child.try_wait().context("Failed to wait for child")? { Some(status) => { @@ -86,7 +108,7 @@ pub fn execute_external_with_redirects( } // Sleep briefly before next check - std::thread::sleep(Duration::from_millis(50)); + pause.sleep(); } } } @@ -224,6 +246,7 @@ pub fn execute_pipeline( let mut final_exit_code = 0; for (idx, mut child) in children.into_iter().enumerate() { + let mut pause = Pause::new(); loop { match child.try_wait().context("Failed to wait for child")? { Some(status) => { @@ -245,7 +268,7 @@ pub fn execute_pipeline( // Don't record modifications - operation interrupted return Ok(130); // 128 + SIGINT } - std::thread::sleep(Duration::from_millis(50)); + pause.sleep(); } } } @@ -303,6 +326,7 @@ pub fn execute_external(program: &str, args: &[String]) -> Result { .context(format!("Failed to spawn: {}", program))?; // Poll for exit while checking for interrupt + let mut pause = Pause::new(); loop { match child.try_wait().context("Failed to wait for child")? { Some(status) => { @@ -323,7 +347,7 @@ pub fn execute_external(program: &str, args: &[String]) -> Result { } // Sleep briefly before next check - std::thread::sleep(Duration::from_millis(50)); + pause.sleep(); } } } diff --git a/impl/rust-cli/src/main.rs b/impl/rust-cli/src/main.rs index ebb288a..3738fba 100644 --- a/impl/rust-cli/src/main.rs +++ b/impl/rust-cli/src/main.rs @@ -186,6 +186,10 @@ fn main() -> Result<()> { print_banner(); } + // An interactive shell leaves with the status of its last command, or the + // one given to `exit`. The one-shot subcommands leave with 0 or an error. + let interactive = matches!(cli.command, Some(Commands::Shell) | None); + match cli.command { Some(Commands::Mkdir { path }) => { commands::mkdir(&mut state, &path, cli.verbose)?; @@ -240,6 +244,10 @@ fn main() -> Result<()> { } } + if interactive { + std::process::exit(state.last_exit_code); + } + Ok(()) } @@ -276,7 +284,9 @@ fn execute_script_content(content: &str, state: &mut state::ShellState) -> Resul Ok(cmd) => { let result = cmd.execute(state)?; match result { - ExecutionResult::Exit => return Ok(()), + // `exit` has already set the status to leave with. Stop + // here; the EXIT trap and the exit itself follow the loop. + ExecutionResult::Exit => break, ExecutionResult::ExternalCommand { exit_code } | ExecutionResult::Return { exit_code } => { state.last_exit_code = exit_code; diff --git a/impl/rust-cli/src/parser.rs b/impl/rust-cli/src/parser.rs index da677ce..f88ab71 100644 --- a/impl/rust-cli/src/parser.rs +++ b/impl/rust-cli/src/parser.rs @@ -258,7 +258,10 @@ pub enum Command { count: usize, show_proofs: bool, }, - Exit, + /// `exit [status]`. The status is kept unexpanded until it runs. + Exit { + code: Option, + }, Quit, // Transactions @@ -3432,7 +3435,14 @@ fn parse_base_command( .unwrap_or(10); Ok(Command::History { count, show_proofs }) } - "exit" => Ok(Command::Exit), + "exit" => { + if args.len() > 1 { + return Err(anyhow!("exit: too many arguments")); + } + Ok(Command::Exit { + code: args.first().cloned(), + }) + } "quit" => Ok(Command::Quit), // Transactions diff --git a/impl/rust-cli/tests/conformance/known_failing.txt b/impl/rust-cli/tests/conformance/known_failing.txt index 75a2bf0..90bb4c6 100644 --- a/impl/rust-cli/tests/conformance/known_failing.txt +++ b/impl/rust-cli/tests/conformance/known_failing.txt @@ -25,15 +25,11 @@ subst-04 # echo $(echo $(echo deep)) pipe-05 # seq 1 5 | awk '{s+=$1} END{print s}' redir-03 # ls /nonexistent 2>/dev/null; echo rc=$? redir-04 # ls /nonexistent > out 2>&1; grep -c . out -logic-03 # [ -d / ] && echo dir -logic-04 # [ 3 -gt 2 ] && [ a = a ] && echo ok logic-05 # [[ abc == a* ]] && echo match logic-07 # ! false && echo negated -if-02 # if [ 1 -eq 2 ]; then echo a; elif [ 1 -eq 1 ]; then echo b; fi if-03 # x=3; if (( x > 2 )); then echo big; fi loop-02 # for i in $(seq 3); do echo n$i; done loop-03 # mkdir zzdir; touch zzdir/p1 zzdir/p2; for f in zzdir/p*; do echo ${f##*/}; done -loop-04 # i=0; while [ $i -lt 3 ]; do i=$((i+1)); done; echo $i loop-05 # for ((i=0;i<3;i++)); do echo $i; done loop-06 # for i in 1 2 3; do [ $i = 2 ] && continue; echo $i; done loop-07 # for i in 1 2 3; do [ $i = 2 ] && break; echo $i; done @@ -58,7 +54,6 @@ builtin-05 # set -e; false; echo notreached builtin-06 # x=1; unset x; echo "[${x}]" builtin-07 # type echo | head -1 | grep -c echo builtin-12 # mkdir zzdir; pushd zzdir >/dev/null; pwd | grep -c zzdir; popd >/dev/null -builtin-15 # exit 7 builtin-16 # command -v ls | grep -c ls builtin-17 # true; echo ${PIPESTATUS[0]} fs-01 # mkdir -p a/b/c && ls a/b @@ -69,8 +64,6 @@ ext-09 # sh -c 'echo "$1"' _ '{p}' ext-10 # echo one > a.txt; echo two > b.txt; cat *.txt loop-10 # for i in a b; do for j in 1 2; do echo $i$j; done; done subst-05 # x=$(echo hi); echo $x -builtin-18 # true; exit 3 -builtin-19 # false; exit fs-09 # touch a b c; ls | wc -l fs-10 # mkdir x y; ls | wc -l rm-02 # rm nosuch; echo rc=$? diff --git a/impl/rust-cli/tests/external_wait_tests.rs b/impl/rust-cli/tests/external_wait_tests.rs new file mode 100644 index 0000000..3619e03 --- /dev/null +++ b/impl/rust-cli/tests/external_wait_tests.rs @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: MPL-2.0 +// Copyright (c) Jonathan D.A. Jewell +//! The shell must not add a fixed wait to every program it runs. +//! +//! It used to check on a running child every 50 ms, so each external command +//! took at least that long: a hundred of them took over five seconds, against +//! about 0.15 s in bash. These tests run the built shell. + +use std::process::{Command, Stdio}; +use std::time::{Duration, Instant}; + +fn vsh(dir: &std::path::Path, script: &str) -> (i32, Duration) { + let started = Instant::now(); + let status = Command::new(env!("CARGO_BIN_EXE_vsh")) + .arg("-r") + .arg(dir) + .arg("-c") + .arg(script) + .current_dir(dir) + .stdin(Stdio::null()) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status() + .expect("start the built shell"); + (status.code().unwrap_or(-1), started.elapsed()) +} + +#[test] +fn a_hundred_short_programs_are_not_each_held_for_a_fixed_interval() { + let dir = tempfile::tempdir().expect("temporary directory"); + let words: Vec = (1..=100).map(|i| i.to_string()).collect(); + let script = format!("for i in {}; do sleep 0; done", words.join(" ")); + + let (code, took) = vsh(dir.path(), &script); + + assert_eq!(code, 0, "the loop should succeed"); + // A fixed 50 ms wait per program gives 5 s. The limit leaves room for a + // slow machine while still failing if such a wait comes back. + assert!( + took < Duration::from_millis(2500), + "100 short programs took {took:?}" + ); +} + +#[test] +fn a_program_that_runs_for_a_while_is_still_waited_for() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let (code, took) = vsh(dir.path(), "sleep 0.3; exit 4"); + + assert_eq!(code, 4, "the status given to exit should be the shell's"); + assert!( + took >= Duration::from_millis(300), + "the shell returned after {took:?}, before the program finished" + ); +} From 78035349bbe53ad908b8f4582a408e6908a93bec Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 13:59:04 +0000 Subject: [PATCH 03/14] fix(builtins): refuse options and extra operands the built-ins do not handle Third slice of the truthful baseline (OD-43). The built-in file commands take fewer forms than the system ones. They took an option for a file name and dropped operands they had no room for, and reported success: mkdir -p a/b made a directory named `-p` touch a b c made only `a` chmod 600 a b changed only `a` rm -rf d tried to remove a file named `-rf` obliterate --force a b destroyed `a` and said nothing about `b` undo many undid one operation One helper in the parser now splits options from operands for mkdir, rmdir, touch, rm, cp, mv, chmod, chown, ln, ls, cd and obliterate. An option the built-in does not know, or an operand beyond what it takes, is an error at parse time: the command does nothing, the status is 1, and a script goes on. `--` ends the options, so a name that starts with `-` can still be given. `chmod -x f` still works: the first argument is the mode. `undo` and `redo` refuse a count they cannot read. Tests: tests/builtin_strictness_tests.rs, nine tests that run the built shell and check both the refusal and that the directory tree, contents and modes are unchanged. Seven of them fail against the old parser (checked). Two existing tests in e2e_script_execution.rs asserted that `mkdir -p ...` parses. They held only because `-p` was taken for the directory name. Both now use the forms the built-in handles, and one asserts the refusal. cargo fmt and clippy with -D warnings clean. cargo test --no-fail-fast: 841 passed, 0 failed, 15 ignored. The battery is unchanged at 66 of 126: these forms still differ from coreutils, but now loudly. Not done: the options themselves (-p, -r, -f and so on). Those belong to the engine's command-line tool (stage S1), checked against coreutils. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- impl/rust-cli/src/parser.rs | 204 ++++++++++---- .../tests/builtin_strictness_tests.rs | 252 ++++++++++++++++++ impl/rust-cli/tests/e2e_script_execution.rs | 18 +- 3 files changed, 413 insertions(+), 61 deletions(-) create mode 100644 impl/rust-cli/tests/builtin_strictness_tests.rs diff --git a/impl/rust-cli/src/parser.rs b/impl/rust-cli/src/parser.rs index f88ab71..4abf833 100644 --- a/impl/rust-cli/src/parser.rs +++ b/impl/rust-cli/src/parser.rs @@ -3256,6 +3256,73 @@ pub fn is_incomplete_control_structure(input: &str) -> bool { block_depth > 0 } +/// Splits the arguments of a built-in command into options and operands, and +/// refuses what the built-in does not handle. +/// +/// A built-in that took an option for a file name, or dropped the operands it +/// had no room for, did something other than what was typed and reported +/// success (`mkdir -p a/b` made a directory named `-p`). Refusing is the safe +/// answer until the option exists. `--` ends the options, so a name that +/// starts with `-` can still be given. +fn builtin_operands( + name: &str, + args: &[String], + known_options: &[&str], + min: usize, + max: usize, + missing: &str, +) -> Result<(Vec, Vec)> { + let mut options = Vec::new(); + let mut operands = Vec::new(); + let mut options_ended = false; + for arg in args { + if !options_ended && arg == "--" { + options_ended = true; + } else if !options_ended && arg.len() > 1 && arg.starts_with('-') { + if !known_options.contains(&arg.as_str()) { + return Err(anyhow!( + "{name}: option '{arg}' is not supported by the built-in {name}; nothing was done" + )); + } + options.push(arg.clone()); + } else { + operands.push(arg.clone()); + } + } + if operands.len() < min { + return Err(anyhow!("{missing}")); + } + if operands.len() > max { + let takes = match max { + 0 => "no operands".to_string(), + 1 => "one operand".to_string(), + n => format!("{n} operands"), + }; + return Err(anyhow!( + "{name}: the built-in {name} takes {takes}, but {} were given; nothing was done", + operands.len() + )); + } + Ok((options, operands)) +} + +/// Parses the count given to `undo` or `redo`: nothing means one. +fn step_count(name: &str, args: &[String]) -> Result { + match args { + [] => Ok(1), + [count] => match count.parse::() { + Ok(n) if n > 0 => Ok(n), + _ => Err(anyhow!( + "{name}: '{count}' is not a count of operations; nothing was done" + )), + }, + _ => Err(anyhow!( + "{name}: takes one count, but {} arguments were given; nothing was done", + args.len() + )), + } +} + fn parse_base_command( cmd: &str, args: Vec, @@ -3264,102 +3331,125 @@ fn parse_base_command( ) -> Result { match cmd { "mkdir" => { - if args.is_empty() { - return Err(anyhow!("mkdir: missing operand")); - } + let (_, operands) = + builtin_operands("mkdir", &args, &[], 1, 1, "mkdir: missing operand")?; Ok(Command::Mkdir { - path: args[0].clone(), + path: operands[0].clone(), redirects, }) } "rmdir" => { - if args.is_empty() { - return Err(anyhow!("rmdir: missing operand")); - } + let (_, operands) = + builtin_operands("rmdir", &args, &[], 1, 1, "rmdir: missing operand")?; Ok(Command::Rmdir { - path: args[0].clone(), + path: operands[0].clone(), redirects, }) } "touch" => { - if args.is_empty() { - return Err(anyhow!("touch: missing operand")); - } + let (_, operands) = + builtin_operands("touch", &args, &[], 1, 1, "touch: missing operand")?; Ok(Command::Touch { - path: args[0].clone(), + path: operands[0].clone(), redirects, }) } "rm" => { - if args.is_empty() { - return Err(anyhow!("rm: missing operand")); - } + let (_, operands) = builtin_operands("rm", &args, &[], 1, 1, "rm: missing operand")?; Ok(Command::Rm { - path: args[0].clone(), + path: operands[0].clone(), redirects, }) } "obliterate" => { // GDPR-style secure deletion. `--force`/`-f` skips confirmation. - let force = args.iter().any(|a| a == "--force" || a == "-f"); - let path = args.iter().find(|a| !a.starts_with('-')).cloned(); - match path { - Some(path) => Ok(Command::Obliterate { - path, - force, - redirects, - }), - None => Err(anyhow!("obliterate: missing operand")), - } + // Exactly one target: an obliteration cannot be undone, so an + // operand that would be passed over in silence is refused. + let (options, operands) = builtin_operands( + "obliterate", + &args, + &["--force", "-f"], + 1, + 1, + "obliterate: missing operand", + )?; + Ok(Command::Obliterate { + path: operands[0].clone(), + force: !options.is_empty(), + redirects, + }) } "cp" => { - if args.len() < 2 { - return Err(anyhow!("cp: missing destination operand")); - } + let (_, operands) = + builtin_operands("cp", &args, &[], 2, 2, "cp: missing destination operand")?; Ok(Command::Cp { - src: args[0].clone(), - dst: args[1].clone(), + src: operands[0].clone(), + dst: operands[1].clone(), redirects, }) } "mv" => { - if args.len() < 2 { - return Err(anyhow!("mv: missing destination operand")); - } + let (_, operands) = + builtin_operands("mv", &args, &[], 2, 2, "mv: missing destination operand")?; Ok(Command::Mv { - src: args[0].clone(), - dst: args[1].clone(), + src: operands[0].clone(), + dst: operands[1].clone(), redirects, }) } "chmod" => { + // The first argument is the mode, which may itself start with `-` + // (`chmod -x f`), so only the option spellings are refused there. + if let Some(first) = args.first() { + if matches!(first.as_str(), "-R" | "-c" | "-f" | "-v") || first.starts_with("--") { + return Err(anyhow!( + "chmod: option '{first}' is not supported by the built-in chmod; nothing was done" + )); + } + } if args.len() < 2 { return Err(anyhow!("chmod: usage: chmod MODE FILE")); } + let (_, operands) = builtin_operands( + "chmod", + &args[1..], + &[], + 1, + 1, + "chmod: usage: chmod MODE FILE", + )?; Ok(Command::Chmod { mode: args[0].clone(), - path: args[1].clone(), + path: operands[0].clone(), redirects, }) } "chown" => { - if args.len() < 2 { - return Err(anyhow!("chown: usage: chown OWNER[:GROUP] FILE")); - } + let (_, operands) = builtin_operands( + "chown", + &args, + &[], + 2, + 2, + "chown: usage: chown OWNER[:GROUP] FILE", + )?; Ok(Command::Chown { - owner: args[0].clone(), - path: args[1].clone(), + owner: operands[0].clone(), + path: operands[1].clone(), redirects, }) } "ln" => { // ln -s target link_name - // Filter out -s flag - let filtered: Vec<_> = args.iter().filter(|a| *a != "-s").cloned().collect(); - if filtered.len() < 2 { - return Err(anyhow!("ln: usage: ln -s target link_name")); - } - if !args.contains(&"-s".to_string()) { + let (options, filtered) = builtin_operands( + "ln", + &args, + &["-s"], + 2, + 2, + "ln: usage: ln -s target link_name", + )?; + if options.is_empty() { return Err(anyhow!("ln: only symbolic links supported (use ln -s)")); } Ok(Command::Ln { @@ -3369,11 +3459,11 @@ fn parse_base_command( }) } "undo" => { - let count = args.first().and_then(|s| s.parse().ok()).unwrap_or(1); + let count = step_count("undo", &args)?; Ok(Command::Undo { count }) } "redo" => { - let count = args.first().and_then(|s| s.parse().ok()).unwrap_or(1); + let count = step_count("redo", &args)?; Ok(Command::Redo { count }) } @@ -3460,18 +3550,22 @@ fn parse_base_command( // Display commands "graph" => Ok(Command::Graph), "proofs" | "proof" => Ok(Command::Proofs), - "ls" => Ok(Command::Ls { - path: args.first().map(|s| s.to_string()), - redirects, - }), + "ls" => { + let (_, operands) = builtin_operands("ls", &args, &[], 0, 1, "ls: missing operand")?; + Ok(Command::Ls { + path: operands.first().cloned(), + redirects, + }) + } "pwd" => Ok(Command::Pwd { redirects }), "cd" => { // cd doesn't support redirections (it's a state change, not output) if !redirects.is_empty() { return Err(anyhow!("cd: redirections not supported for this command")); } + let (_, operands) = builtin_operands("cd", &args, &[], 0, 1, "cd: missing operand")?; Ok(Command::Cd { - path: args.first().map(|s| s.to_string()), + path: operands.first().cloned(), }) } diff --git a/impl/rust-cli/tests/builtin_strictness_tests.rs b/impl/rust-cli/tests/builtin_strictness_tests.rs new file mode 100644 index 0000000..982f900 --- /dev/null +++ b/impl/rust-cli/tests/builtin_strictness_tests.rs @@ -0,0 +1,252 @@ +// SPDX-License-Identifier: MPL-2.0 +// Copyright (c) Jonathan D.A. Jewell +//! A built-in command refuses what it does not handle, and then does nothing. +//! +//! The built-in file commands take fewer forms than the system ones. They used +//! to take an option for a file name and to drop the operands they had no room +//! for, and report success: `mkdir -p a/b` made a directory named `-p`, and +//! `touch a b c` made only `a`. Each test here runs the built shell and checks +//! both the refusal and that the directory is as it was. + +use std::collections::BTreeSet; +use std::fs; +use std::os::unix::fs::PermissionsExt; +use std::path::Path; +use std::process::{Command, Stdio}; + +struct Run { + code: i32, + stdout: String, + stderr: String, +} + +fn vsh(dir: &Path, script: &str) -> Run { + let output = Command::new(env!("CARGO_BIN_EXE_vsh")) + .arg("-r") + .arg(dir) + .arg("-c") + .arg(script) + .current_dir(dir) + .env("NO_COLOR", "1") + .env_remove("RUST_BACKTRACE") + .stdin(Stdio::null()) + .output() + .expect("start the built shell"); + Run { + code: output.status.code().unwrap_or(-1), + stdout: String::from_utf8_lossy(&output.stdout).into_owned(), + stderr: String::from_utf8_lossy(&output.stderr).into_owned(), + } +} + +/// Every path under `dir`, relative to it, without the shell's own state files. +fn tree(dir: &Path) -> BTreeSet { + fn walk(base: &Path, at: &Path, found: &mut BTreeSet) { + for entry in fs::read_dir(at).expect("read directory") { + let path = entry.expect("directory entry").path(); + let name = path + .strip_prefix(base) + .expect("inside the base") + .to_string_lossy() + .into_owned(); + if name.starts_with(".vsh") { + continue; + } + let is_dir = fs::symlink_metadata(&path) + .expect("metadata") + .file_type() + .is_dir(); + found.insert(name); + if is_dir { + walk(base, &path, found); + } + } + } + let mut found = BTreeSet::new(); + walk(dir, dir, &mut found); + found +} + +/// A directory holding files `a` and `b`, and a directory `d` holding `inner`. +fn fixture() -> tempfile::TempDir { + let dir = tempfile::tempdir().expect("temporary directory"); + fs::write(dir.path().join("a"), "alpha\n").expect("write a"); + fs::write(dir.path().join("b"), "beta\n").expect("write b"); + fs::create_dir(dir.path().join("d")).expect("create d"); + fs::write(dir.path().join("d/inner"), "inner\n").expect("write inner"); + dir +} + +/// Runs `script`, which must be refused, and checks nothing changed. +fn refused(script: &str, expected_in_message: &str) { + let dir = fixture(); + let before = tree(dir.path()); + + let run = vsh(dir.path(), script); + + assert_ne!(run.code, 0, "`{script}` should not report success"); + assert!( + run.stderr.contains(expected_in_message), + "`{script}` should say {expected_in_message:?}; it said {:?}", + run.stderr + ); + assert_eq!( + tree(dir.path()), + before, + "`{script}` was refused but still changed the directory" + ); + assert_eq!( + fs::read_to_string(dir.path().join("a")).expect("read a"), + "alpha\n", + "`{script}` changed the contents of a" + ); +} + +#[test] +fn an_option_is_never_taken_for_a_file_name() { + for script in [ + "mkdir -p x/y", + "rmdir -p d", + "touch -c x", + "rm -rf d", + "rm -f a", + "cp -r d e", + "cp -p a c", + "mv -f a b", + "chown -R 0:0 d", + "ln -sf a l", + "ls -la", + "cd -P d", + ] { + refused(script, "is not supported by the built-in"); + } +} + +#[test] +fn chmod_refuses_its_options_but_takes_a_mode_that_starts_with_a_dash() { + refused("chmod -R 700 d", "is not supported by the built-in"); + refused( + "chmod --recursive 700 d", + "is not supported by the built-in", + ); + + let dir = fixture(); + let file = dir.path().join("a"); + fs::set_permissions(&file, fs::Permissions::from_mode(0o755)).expect("set mode"); + + let run = vsh(dir.path(), "chmod -x a"); + + assert_eq!(run.code, 0, "chmod -x a failed: {}", run.stderr); + let mode = fs::metadata(&file).expect("metadata").permissions().mode() & 0o777; + assert_eq!(mode, 0o644, "the execute bits should be gone"); +} + +#[test] +fn operands_with_no_place_to_go_are_never_dropped() { + for script in [ + "mkdir x y", + "rmdir d e", + "touch x y z", + "rm a b", + "cp a b d", + "mv a b d", + "chown 0:0 a b", + "ln -s a l m", + "ls d d", + "cd d d", + ] { + refused(script, "were given; nothing was done"); + } +} + +#[test] +fn chmod_does_not_change_the_first_file_when_given_two() { + let dir = fixture(); + let before = fs::metadata(dir.path().join("a")) + .expect("metadata") + .permissions() + .mode(); + + let run = vsh(dir.path(), "chmod 600 a b"); + + assert_ne!(run.code, 0); + let after = fs::metadata(dir.path().join("a")) + .expect("metadata") + .permissions() + .mode(); + assert_eq!(after, before, "a refused chmod must change no mode"); +} + +#[test] +fn obliterate_refuses_a_second_target_and_an_unknown_option() { + refused("obliterate --force a b", "were given; nothing was done"); + refused("obliterate --frce a", "is not supported by the built-in"); +} + +#[test] +fn undo_and_redo_refuse_a_count_they_cannot_read() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh(dir.path(), "touch kept; undo many"); + + assert_ne!(run.code, 0); + assert!( + run.stderr.contains("is not a count of operations"), + "undo said {:?}", + run.stderr + ); + assert!( + dir.path().join("kept").exists(), + "`undo many` must not undo anything" + ); + + let run = vsh(dir.path(), "undo 1 2"); + assert_ne!(run.code, 0); + assert!(dir.path().join("kept").exists()); + + let run = vsh(dir.path(), "redo 0"); + assert_ne!(run.code, 0); +} + +#[test] +fn a_double_dash_lets_a_name_start_with_a_dash() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh(dir.path(), "mkdir -- -p"); + + assert_eq!(run.code, 0, "mkdir -- -p failed: {}", run.stderr); + assert!(dir.path().join("-p").is_dir()); +} + +#[test] +fn a_refusal_does_not_end_the_script() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh(dir.path(), "mkdir -p x/y; echo after"); + + assert!( + run.stdout.contains("after"), + "the script should go on after a refused command; stdout was {:?}", + run.stdout + ); +} + +#[test] +fn the_forms_the_built_ins_do_handle_still_work() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh( + dir.path(), + "mkdir d; touch d/f; cp d/f g; mv g h; chmod 600 h; ln -s h l; rm d/f; rmdir d; ls", + ); + + assert_eq!(run.code, 0, "stderr: {}", run.stderr); + let expected: BTreeSet = ["h", "l"].iter().map(|name| name.to_string()).collect(); + assert_eq!(tree(dir.path()), expected); + let mode = fs::metadata(dir.path().join("h")) + .expect("metadata") + .permissions() + .mode() + & 0o777; + assert_eq!(mode, 0o600); +} diff --git a/impl/rust-cli/tests/e2e_script_execution.rs b/impl/rust-cli/tests/e2e_script_execution.rs index 3927314..34e177c 100644 --- a/impl/rust-cli/tests/e2e_script_execution.rs +++ b/impl/rust-cli/tests/e2e_script_execution.rs @@ -143,7 +143,9 @@ mkdir test_dir let temp = TempDir::new().unwrap(); let script_path = temp.path().join("build.sh"); - let script = "mkdir -p src/main\ntouch src/main/lib.rs\n"; + // The built-in mkdir has no `-p` (it is refused, not taken for a name), + // so the script makes each directory in turn. + let script = "mkdir src\nmkdir src/main\ntouch src/main/lib.rs\n"; fs::write(&script_path, script).unwrap(); let content = fs::read_to_string(&script_path).unwrap(); @@ -297,16 +299,20 @@ mkdir test_dir let root = temp.path().to_str().unwrap(); let mut state = ShellState::new(root).unwrap(); - // Parse: mkdir -p build (conditional create) + // The built-in mkdir has no `-p`. It used to take `-p` for the name of + // the directory; it now refuses the option (see + // `builtin_strictness_tests.rs`). let result = parse_command("mkdir -p build"); - assert!(result.is_ok()); + assert!(result.is_err()); - // Execute + // Without the option it parses and runs. + assert!(parse_command("mkdir build").is_ok()); let exec_result = mkdir(&mut state, "build", false); assert!(exec_result.is_ok()); - // Try again (should still work due to -p flag semantics) - let _exec_result2 = mkdir(&mut state, "build", false); + // A second attempt fails, because the directory is there. + let exec_result2 = mkdir(&mut state, "build", false); + assert!(exec_result2.is_err()); } #[test] From 4489143d237ee6c4e770fca5f3b2b7c84e190c47 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 14:02:21 +0000 Subject: [PATCH 04/14] fix(help): `help` is a command; in-product texts say what the proofs cover MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fourth slice of the truthful baseline (OD-43). help - The interactive banner says "Type 'help'", and the default interactive loop answered "help: Command not found". `help [command]` is now a parsed command, so it works at the prompt and with -c. - The list shows every command that exists, with a summary where there is one. It no longer advertises `help -v` or `man vsh-`, which do not exist, and it states the limit of the built-in file commands. Texts that said more than the proof ledger allows - `replay` and `diff` printed "all proven reversible" and "All reversals proven ✓". The theorems are about the model, and the link from the model to this code is tested, not proved. They now name the theorem as being about the model and show no tick. - `proofs` printed "✓ Zig FFI layer checks preconditions". That layer is not connected to the shell. The trust model now says so and lists the model-to-code link as open. - `obliterate` called its three overwrites a "NIST SP 800-88 Purge pattern". An overwrite with ordinary writes is the Clear level at most. The hardware-erase messages keep "Purge", where it is correct. - The manual-page text said an operation "is formally verified for correctness". Tests: tests/help_tests.rs (four tests, built shell). cargo fmt and clippy with -D warnings clean. cargo test --no-fail-fast: 845 passed, 0 failed, 15 ignored. `help` also checked by hand in an interactive session. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- impl/rust-cli/src/commands.rs | 25 ++++--- impl/rust-cli/src/commands/secure_deletion.rs | 9 ++- impl/rust-cli/src/executable.rs | 6 ++ impl/rust-cli/src/help.rs | 58 ++++++++++------ impl/rust-cli/src/parser.rs | 11 ++++ impl/rust-cli/src/proof_refs.rs | 18 +++-- impl/rust-cli/tests/help_tests.rs | 66 +++++++++++++++++++ 7 files changed, 151 insertions(+), 42 deletions(-) create mode 100644 impl/rust-cli/tests/help_tests.rs diff --git a/impl/rust-cli/src/commands.rs b/impl/rust-cli/src/commands.rs index 4f91625..fa7e538 100644 --- a/impl/rust-cli/src/commands.rs +++ b/impl/rust-cli/src/commands.rs @@ -2262,7 +2262,7 @@ pub fn explain_command(cmd: &crate::parser::Command, state: &ShellState) -> Resu println!(" {}", "Inverse operation:".bright_yellow().bold()); println!(" fs' ─── {} ───▶ fs", format!("{}", inv).bright_red(),); println!( - " Proof: {} {}", + " Theorem (model): {} {}", proof_ref .as_ref() .map_or("(none)", |p| p.description) @@ -2515,10 +2515,11 @@ pub fn diff_state(state: &ShellState, target_op: usize) -> Result<()> { ); let composition_proof = crate::proof_refs::COMPOSITION_REVERSIBLE; + // The theorem is about the model. Whether this code follows the model is + // tested, not proved (EXPLAINME.adoc, proof ledger), so no tick is shown. println!( - " All reversals proven: {} {}", - composition_proof.theorem.bright_cyan(), - "✓".bright_green() + " Reversal theorem (about the model): {}", + composition_proof.theorem.bright_cyan() ); Ok(()) } @@ -2564,11 +2565,7 @@ pub fn replay(state: &ShellState, start: usize, end: usize) -> Result<()> { format!("_{}", idx) }; println!(" State: fs{} ──▶ fs_{}", from_state, idx + 1); - println!( - " Proof: {} {}", - proof.theorem.bright_cyan(), - "✓".bright_green() - ); + println!(" Theorem (model): {}", proof.theorem.bright_cyan()); println!(" [{}] {}%", bar.bright_green(), (step * 100) / total); println!(); @@ -2577,16 +2574,18 @@ pub fn replay(state: &ShellState, start: usize, end: usize) -> Result<()> { } println!( - "Replay complete. {}: {} operations, all proven reversible.", + "Replay complete. {}: {} operations.", "Total".bright_white().bold(), total ); + println!( + "Each has a reversal theorem about the model; the link from the model to this code is tested, not proved." + ); let composition = crate::proof_refs::COMPOSITION_REVERSIBLE; println!( - "Composition theorem: {} {}", - composition.theorem.bright_cyan(), - "✓".bright_green() + "Composition theorem (model): {}", + composition.theorem.bright_cyan() ); // Show full undo path diff --git a/impl/rust-cli/src/commands/secure_deletion.rs b/impl/rust-cli/src/commands/secure_deletion.rs index 3737390..1716cf9 100644 --- a/impl/rust-cli/src/commands/secure_deletion.rs +++ b/impl/rust-cli/src/commands/secure_deletion.rs @@ -158,7 +158,11 @@ fn overwrite_pass(file: &mut std::fs::File, file_size: u64, pattern: Pattern) -> Ok(()) } -/// Perform 3-pass NIST SP 800-88 Purge-style secure overwrite. +/// Overwrite the file three times. +/// +/// In NIST SP 800-88 terms an overwrite with ordinary write calls is the +/// *Clear* level at most. *Purge* needs a device sanitize command or +/// cryptographic erase (see `secure_erase`). /// /// Pass order matches the task spec (random / zeros / final pattern), chosen /// because the final pattern is a known constant — useful for verifying that @@ -307,7 +311,8 @@ pub fn obliterate(state: &mut ShellState, path: &str, verbose: bool, force: bool eprintln!(" Size: {} bytes", file_size.to_string().bright_yellow()); eprintln!(); eprintln!("This will:"); - eprintln!(" 1. Overwrite with 3-pass NIST SP 800-88 Purge pattern (random, 0x00, 0xFF)"); + eprintln!(" 1. Overwrite the file three times (random, 0x00, 0xFF)."); + eprintln!(" In NIST SP 800-88 terms this is the Clear level at most, not Purge."); eprintln!(" 2. Delete the file"); eprintln!(" 3. Reduce ordinary file-level recovery opportunities"); eprintln!(" (CoW filesystems like btrfs/ZFS/APFS and SSDs with FTL"); diff --git a/impl/rust-cli/src/executable.rs b/impl/rust-cli/src/executable.rs index ca8841d..36b0584 100644 --- a/impl/rust-cli/src/executable.rs +++ b/impl/rust-cli/src/executable.rs @@ -278,6 +278,11 @@ impl ExecutableCommand for Command { Ok(ExecutionResult::Success) } + Command::Help { topic } => { + crate::help::display_help(topic.as_deref(), crate::help::HelpTier::Verbose)?; + Ok(ExecutionResult::Success) + } + // Exit commands Command::Exit { code } => { // `exit` alone keeps the status of the last command. Only the low @@ -1268,6 +1273,7 @@ impl ExecutableCommand for Command { Command::Rollback => "rollback".to_string(), Command::Graph => "graph".to_string(), Command::Proofs => "proofs".to_string(), + Command::Help { .. } => "help".to_string(), Command::Ls { path, .. } => { if let Some(p) = path { format!("ls {}", p) diff --git a/impl/rust-cli/src/help.rs b/impl/rust-cli/src/help.rs index b2e30af..4308965 100644 --- a/impl/rust-cli/src/help.rs +++ b/impl/rust-cli/src/help.rs @@ -200,7 +200,9 @@ impl HelpEntry { if let Some(op_type) = self.proof_operation { output.push_str("FORMAL VERIFICATION\n"); let proof_ref = proof_refs::ProofReference::for_operation(op_type); - output.push_str(" This operation is formally verified for correctness.\n\n"); + output.push_str( + " This operation has a reversal theorem about the model. Whether the\n code follows the model is tested, not proved.\n\n", + ); output.push_str(&format!(" Theorem: {}\n", proof_ref.theorem)); output.push_str(&format!( " Description: {}\n\n", @@ -279,31 +281,51 @@ fn display_command_list() -> Result<()> { // Group by category let categories = vec![ - ("Navigation", vec!["cd", "pwd"]), - ("File Operations", vec!["mkdir", "rmdir", "touch", "rm"]), - ("History & Undo", vec!["undo", "redo", "history"]), + ("Navigation", vec!["cd", "pwd", "ls"]), + ( + "File Operations", + vec![ + "mkdir", "rmdir", "touch", "rm", "cp", "mv", "ln", "chmod", "chown", + ], + ), + ( + "History & Undo", + vec![ + "undo", + "redo", + "history", + "explain", + "checkpoint", + "restore", + "diff", + "replay", + "graph", + ], + ), ("Transactions", vec!["begin", "commit", "rollback"]), - ("Utilities", vec!["help", "exit"]), + ("Irreversible", vec!["obliterate"]), + ("Utilities", vec!["help", "proofs", "jobs", "exit"]), ]; for (category, commands) in categories { println!("{}", category.bright_yellow()); for cmd in commands { - if let Some(entry) = get_help(cmd) { - println!(" {} - {}", cmd.bright_green(), entry.summary); + // A command without a detailed entry is still listed, so the list + // does not suggest it is missing. + match get_help(cmd) { + Some(entry) => println!(" {} - {}", cmd.bright_green(), entry.summary), + None => println!(" {}", cmd.bright_green()), } } println!(); } - println!("Run {} for command details", "help ".bright_cyan()); println!( - "Run {} for detailed help with examples", - "help -v ".bright_cyan() + "Run {} for details, where a command has them.", + "help ".bright_cyan() ); println!( - "Run {} for full manual page", - "man vsh-".bright_cyan() + "The built-in file commands take one operand each (two for cp, mv and ln -s) and no options." ); Ok(()) @@ -454,12 +476,10 @@ static HELP_DATABASE: &[HelpEntry] = &[ HelpEntry { name: "help", summary: "Display help information", - usage: "help [command] [-v]", - description: "Shows help at three levels:\n\ + usage: "help [command]", + description: "Shows help at two levels:\n\ - help : List all commands\n\ - - help : Quick reference\n\ - - help -v : Detailed help with examples\n\ - - man vsh- : Full manual page", + - help : Usage and examples, where the command has them", examples: &[ Example { description: "List all commands", @@ -469,10 +489,6 @@ static HELP_DATABASE: &[HelpEntry] = &[ description: "Quick help for mkdir", command: "help mkdir", }, - Example { - description: "Detailed help with examples", - command: "help -v mkdir", - }, ], related: &[], proof_operation: None, diff --git a/impl/rust-cli/src/parser.rs b/impl/rust-cli/src/parser.rs index 4abf833..a53e0bf 100644 --- a/impl/rust-cli/src/parser.rs +++ b/impl/rust-cli/src/parser.rs @@ -274,6 +274,10 @@ pub enum Command { // Display commands Graph, Proofs, + /// `help [command]` + Help { + topic: Option, + }, Ls { path: Option, redirects: Vec, @@ -3550,6 +3554,13 @@ fn parse_base_command( // Display commands "graph" => Ok(Command::Graph), "proofs" | "proof" => Ok(Command::Proofs), + "help" | "?" => { + let (_, operands) = + builtin_operands("help", &args, &[], 0, 1, "help: missing operand")?; + Ok(Command::Help { + topic: operands.first().cloned(), + }) + } "ls" => { let (_, operands) = builtin_operands("ls", &args, &[], 0, 1, "ls: missing operand")?; Ok(Command::Ls { diff --git a/impl/rust-cli/src/proof_refs.rs b/impl/rust-cli/src/proof_refs.rs index c735bc3..4667896 100644 --- a/impl/rust-cli/src/proof_refs.rs +++ b/impl/rust-cli/src/proof_refs.rs @@ -322,14 +322,21 @@ pub fn print_verification_summary() { println!("{}", "Trust Model:".bright_yellow()); println!( - " {} Formal proofs verified by proof assistant kernels", + " {} Theorems about the model are checked by proof assistants", "✓".bright_green() ); println!( - " {} Zig FFI layer checks preconditions", + " {} The link from the model to this Rust code is tested, not proved", + "○".bright_yellow() + ); + println!( + " {} The Zig FFI layer is not connected to this shell", + "○".bright_yellow() + ); + println!( + " {} The shell keeps the operation history that undo uses", "✓".bright_green() ); - println!(" {} Rust CLI maintains undo stack", "✓".bright_green()); println!( " {} POSIX semantics assumed correct (OS trust)", "○".bright_yellow() @@ -337,9 +344,8 @@ pub fn print_verification_summary() { println!(); println!("{}", "Verification Gap:".bright_red()); - println!(" The FFI layer is NOT mechanically verified."); - println!(" It implements precondition checks derived from proofs."); - println!(" Full end-to-end verification requires:"); + println!(" No proof connects the model to the code that runs."); + println!(" Closing that gap requires:"); println!(" - Formal POSIX semantics model"); println!(" - Refinement proof from abstract to concrete"); println!(" - Verified compilation chain"); diff --git a/impl/rust-cli/tests/help_tests.rs b/impl/rust-cli/tests/help_tests.rs new file mode 100644 index 0000000..e3782f7 --- /dev/null +++ b/impl/rust-cli/tests/help_tests.rs @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: MPL-2.0 +// Copyright (c) Jonathan D.A. Jewell +//! `help` is a command of the shell. +//! +//! The interactive banner tells the user to type `help`. The default +//! interactive loop answered "help: Command not found". These tests run the +//! built shell. + +use std::process::{Command, Stdio}; + +fn vsh(script: &str) -> (i32, String, String) { + let dir = tempfile::tempdir().expect("temporary directory"); + let output = Command::new(env!("CARGO_BIN_EXE_vsh")) + .arg("-r") + .arg(dir.path()) + .arg("-c") + .arg(script) + .current_dir(dir.path()) + .env("NO_COLOR", "1") + .env_remove("RUST_BACKTRACE") + .stdin(Stdio::null()) + .output() + .expect("start the built shell"); + ( + output.status.code().unwrap_or(-1), + String::from_utf8_lossy(&output.stdout).into_owned(), + String::from_utf8_lossy(&output.stderr).into_owned(), + ) +} + +#[test] +fn help_lists_the_commands() { + let (code, stdout, stderr) = vsh("help"); + + assert_eq!(code, 0, "stderr: {stderr}"); + for name in ["mkdir", "undo", "rollback", "obliterate", "explain", "help"] { + assert!(stdout.contains(name), "`help` should list {name}: {stdout}"); + } +} + +#[test] +fn help_for_one_command_describes_it() { + let (code, stdout, stderr) = vsh("help mkdir"); + + assert_eq!(code, 0, "stderr: {stderr}"); + assert!(stdout.contains("mkdir"), "stdout: {stdout}"); +} + +#[test] +fn help_for_an_unknown_command_says_so() { + let (code, _, stderr) = vsh("help nosuchcommand"); + + assert_ne!(code, 0); + assert!( + stderr.contains("No help available"), + "stderr should say there is no help: {stderr}" + ); +} + +#[test] +fn help_does_not_promise_things_that_are_not_there() { + let (_, stdout, _) = vsh("help"); + + assert!(!stdout.contains("man vsh-"), "there are no manual pages"); + assert!(!stdout.contains("help -v"), "there is no -v option"); +} From 69f7934f478004ac66beb7a9feea0bff5e0780bf Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 14:04:01 +0000 Subject: [PATCH 05/14] docs: feature claims match what the built shell does (gate G0) Fifth slice of the truthful baseline (OD-43). Documents only. The root CLAUDE.md listed as working a number of features that the built shell does not deliver: glob expansion, `[`, arithmetic and command substitution in arguments, quoting, `export`, job control. They exist in the source and pass unit tests, and fail when the shell is run. - CLAUDE.md "What Actually Works" now has two lists, both from the battery (66 of 126 command lines match bash on 2026-11-17): what works through the built shell, and what is in the source but does not. "Immediate Priorities" no longer lists POSIX items: they are parked by owner ruling OD-42, and the forward order is S0, the engine, then the capture experiment. Two rules are added under "What NOT to Do". - README.adoc "Shell Features": a note says how the list is measured, and eight ticked boxes become partial with the limit stated. - ROADMAP.adoc: the POSIX section is marked parked. The README's existing render error at line 43 (a level-0 section) is on main and is not touched here. Test counts and other stale figures in these files are left for the route plan's Phase F. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- CLAUDE.md | 89 ++++++++++++++++++++++++++++++---------------------- README.adoc | 25 ++++++++++----- ROADMAP.adoc | 5 +++ 3 files changed, 74 insertions(+), 45 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index b6a2c50..36b6c57 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -22,31 +22,44 @@ Opus audit session that fixed all broken tests, real bugs, and documentation. ### What Actually Works (v0.9.0) -The Rust CLI is a functional interactive shell with these features: - -- Built-in filesystem operations (mkdir, rmdir, touch, rm) with undo/redo -- External command execution via PATH lookup -- Reversible file operations (`cp`, `mv`, `ln -s`) with formal proofs -- Unix pipelines (`cmd1 | cmd2 | cmd3`) -- I/O redirections (`>`, `>>`, `<`, `2>`, `2>>`, `&>`, `2>&1`) -- Process substitution (`<(cmd)` and `>(cmd)`) -- Arithmetic expansion (`$((expr))`) -- Here documents (`<`, `>>`, `<` and here strings (`<</dev/null`) +- A failing built-in ends a `-c` script +- Ctrl-Z does not suspend a running program + +Not exercised by the battery: process substitution, command correction, the pager. ### Proof policy (normative, estate-wide) @@ -271,22 +284,21 @@ vsh::parser::expand_variables(&input, &state); ### Immediate Priorities -1. **Word splitting in external command arguments** (requires quote-context threading) -2. **Alias expansion in pipe segments** (currently only first command in a pipeline) -3. **Echidna integration** for automated property-based verification -4. **`~user` tilde expansion** (requires getpwnam/NSS lookup) +**The shell-language items that used to be listed here are parked** (owner ruling OD-42, +2026-11-17; recorded in `docs/plans/VSH-ROUTE-PLAN-2026-11-04.adoc` on valence-shell#230, +with the measurements in `docs/plans/VSH-PRODUCT-REVIEW-2026-11-17.adoc`). The engine is +the product: reversible operations, the journal, containment, the record, and the proofs. +The interpreter is one front end and comes last. -### This Week +Forward order: -1. Implement shell functions (`func() { ... }`) -2. Shell script execution (`.sh` files) -3. Set up Echidna property-based validation pipeline +1. **S0, the truthful baseline** (gate G0): what is claimed equals what the battery shows +2. **The engine** (route plan phases B, C3 to C5, D, E): runtime holes, state outside the + tree, a store for old file contents, the spec spine, contracts on the Rust core +3. **The capture experiment** (undo for any command), which decides the first front end -### This Month - -1. Implement shell functions (`func() { ... }`) -2. Shell script execution (`.sh` files) -3. Begin Idris2 extraction path for v2.0 +Do not spend effort on POSIX coverage of the interpreter (word splitting, `~user`, alias +expansion in pipelines) unless the owner asks for it. ### What NOT to Do @@ -294,6 +306,9 @@ vsh::parser::expand_variables(&input, &state); - Do NOT inflate version numbers - Do NOT add features before closing proof holes - Do NOT trust Sonnet's commit messages or claims without verification +- Do NOT add a case to `tests/conformance/known_failing.txt` to get a green build: fix the + shell, or correct a wrong expectation and say why +- Do NOT tick a feature as working unless a test that runs the built shell shows it ## Building and Testing diff --git a/README.adoc b/README.adoc index 4211432..cfad6b0 100644 --- a/README.adoc +++ b/README.adoc @@ -50,19 +50,28 @@ Valence Shell is a formally verified shell with proven reversibility guarantees. * [x] MAA Framework: Mutually Assured Accountability with audit trails == Shell Features (v0.9.0) -* [x] Unix Pipelines: Multi-stage pipelines (`cmd1 | cmd2 | cmd3`) -* [x] I/O Redirections: All POSIX redirections (`>`, `>>`, `<`, `2>`, `2>>`, `&>`, `2>&1`) -* [x] Process Substitution: `<(cmd)` and `>(cmd)` with FIFO implementation -* [x] Arithmetic Expansion: `$expr` with full operator support -* [x] Here Documents: `<`, `>>`, `<` and here strings work. A redirection on a built-in command is resolved inside the session root, so `2>/dev/null` fails there +* [~] Process Substitution: `<(cmd)` and `>(cmd)` are in the source; not exercised by the battery +* [~] Arithmetic Expansion: `$((expr))` works as the whole value of an assignment, not inside an argument +* [~] Here Documents: here strings (`<<` describes one; there are no manual pages * [x] 794 Tests Passing: Unit, correspondence, integration, property, security, doctests (0 failures, 14 ignored) — `tested-only` in the proof ledger, not proofs == Implementation diff --git a/ROADMAP.adoc b/ROADMAP.adoc index a708b79..3bc791a 100644 --- a/ROADMAP.adoc +++ b/ROADMAP.adoc @@ -86,6 +86,11 @@ integration), #56/#57/#58 (the three admits surfaced by #55). == v0.10.x — POSIX Coverage +[IMPORTANT] +==== +*Parked* by owner ruling OD-42 (2026-11-17; recorded in `docs/plans/VSH-ROUTE-PLAN-2026-11-04.adoc` on valence-shell#230). The engine is the product and the shell interpreter is one front end, which comes last. What the interpreter does today is measured by `impl/rust-cli/tests/conformance_battery.rs` (66 of 126 command lines match bash), not by the list below, parts of which are out of date. +==== + Most critical missing POSIX features, ranked by impact (per CLAUDE.md and `docs/POSIX_COMPLIANCE.md`): From 0305e3ab38f77dad5a734d8602ab4141998068b1 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 15:01:28 +0000 Subject: [PATCH 06/14] fix(shell): nine review findings on the baseline: refusals, exit, help and the battery Ten findings from the review of 69f7934. Nine are valid and fixed here. The tenth is not: `cd -` works (a lone `-` is an operand, and `test_parse_cd_dash` passes). Built-ins - An operand that only reads as an option once its variable is expanded got past the refusal: `opt=-p; mkdir $opt` made a directory named `-p`. Each built-in's operands are now checked again after expansion, and refused the same way. A word after `--` is still an operand, a `chmod` mode may still start with `-`, and a lone `-` still means the previous directory. Like a refusal found while parsing, it fails that command only and the script goes on. - `exit` with more than one operand printed an error and the script carried on: `vsh -c 'exit 1 2; touch after'` created `after` and ended with status 0. A script now stops there with status 1, as under bash 5.2.21 and dash (both run here). Typed at a prompt it is reported and the shell stays open. - The two interactive loops left the exit status as it was after a command that succeeded, so `false; echo ok; exit` ended the shell with status 1. A command that succeeds now sets 0, and one that fails sets 1. - `help ` still ended with "Run man vsh-", and there are no manual pages. The line is gone. - Help was handed to a pager even when the output was captured or piped. With `more` and no `less` the pager was started with options it does not take and the text was lost. Output that is not going to a terminal is now written as it is. The battery - Case rm-05 matched for the wrong reason. It set up its files with `touch a b`, which the built-in refuses, so nothing was created and the `rm` had nothing to remove. It now creates the files one at a time, and fails, as it should: the built-in `rm` takes one operand. It is added to known_failing.txt. That file may otherwise only shrink; this entry is there because the case was wrong, not because the shell got worse. The other cases that match were checked for the same fault; none has it. - One case added, `exit 1 2; echo after`, recorded from bash. It matches. - The test read the shell's output only after the shell had finished, so a case that wrote more than a pipe holds would have been counted as not finishing. Output is now read while the shell runs, and a case that does not finish is stopped together with its process group. - A line without an `id` was dropped without a word. It is now an error. - The test is built on Linux only. Its expectations are from Linux, and it would have failed every macOS job in the test matrix. The battery now holds 127 cases and 66 match (61 listed as failing). Tests: 13 in builtin_strictness_tests.rs (4 new), 5 in help_tests.rs (1 new, 1 widened), 2 unit tests in each interactive loop, 5 new in conformance_battery.rs. Five mutation runs: each fix taken out fails its test. cargo fmt and clippy with -D warnings clean. cargo test --no-fail-fast: 859 passed, 0 failed, 15 ignored. Not changed: a built-in that fails for another reason (`mkdir` on a name that exists) still ends a script. That is one of the listed failures and belongs to the interpreter work that OD-42 parks. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- CLAUDE.md | 2 +- README.adoc | 2 +- ROADMAP.adoc | 2 +- impl/rust-cli/src/enhanced_repl.rs | 49 ++++- impl/rust-cli/src/executable.rs | 97 +++++++-- impl/rust-cli/src/help.rs | 6 - impl/rust-cli/src/main.rs | 13 +- impl/rust-cli/src/pager.rs | 8 +- impl/rust-cli/src/parser.rs | 32 ++- impl/rust-cli/src/repl.rs | 49 ++++- impl/rust-cli/src/state.rs | 5 + .../tests/builtin_strictness_tests.rs | 65 ++++++ impl/rust-cli/tests/conformance/README.adoc | 18 +- impl/rust-cli/tests/conformance/battery.jsonl | 3 +- .../tests/conformance/known_failing.txt | 1 + impl/rust-cli/tests/conformance_battery.rs | 187 ++++++++++++++---- impl/rust-cli/tests/help_tests.rs | 52 ++++- 17 files changed, 510 insertions(+), 81 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 36b6c57..b459347 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -23,7 +23,7 @@ Opus audit session that fixed all broken tests, real bugs, and documentation. ### What Actually Works (v0.9.0) **Measured, not claimed (2026-11-17, gate G0).** `impl/rust-cli/tests/conformance_battery.rs` -starts the built shell on 126 everyday command lines and compares the result with bash: +starts the built shell on 127 everyday command lines and compares the result with bash: **66 match**. The cases that fail are listed in `impl/rust-cli/tests/conformance/known_failing.txt`. That list is a ratchet: it may only shrink, and a case may not be added to it to make a build pass. diff --git a/README.adoc b/README.adoc index cfad6b0..75b8e5b 100644 --- a/README.adoc +++ b/README.adoc @@ -53,7 +53,7 @@ Valence Shell is a formally verified shell with proven reversibility guarantees. [NOTE] ==== -*Measured, not claimed.* `impl/rust-cli/tests/conformance_battery.rs` starts the built shell on 126 everyday command lines and compares the result with bash. On 2026-11-17, 66 match. The cases that fail are listed in `impl/rust-cli/tests/conformance/known_failing.txt`, which may only shrink. Below, `[x]` means it works through the built shell, `[~]` means in part, and the limit is stated. +*Measured, not claimed.* `impl/rust-cli/tests/conformance_battery.rs` starts the built shell on 127 everyday command lines and compares the result with bash. On 2026-11-17, 66 match. The cases that fail are listed in `impl/rust-cli/tests/conformance/known_failing.txt`, which may only shrink. Below, `[x]` means it works through the built shell, `[~]` means in part, and the limit is stated. ==== * [~] Built-in file commands with undo: `mkdir`, `rmdir`, `touch`, `rm`, `cp`, `mv`, `ln -s`, `chmod`, `chown`, in their plain forms only (one operand each, two for `cp`, `mv` and `ln -s`; no options). An option or an extra operand is refused and nothing is done diff --git a/ROADMAP.adoc b/ROADMAP.adoc index 3bc791a..67123c2 100644 --- a/ROADMAP.adoc +++ b/ROADMAP.adoc @@ -88,7 +88,7 @@ integration), #56/#57/#58 (the three admits surfaced by #55). [IMPORTANT] ==== -*Parked* by owner ruling OD-42 (2026-11-17; recorded in `docs/plans/VSH-ROUTE-PLAN-2026-11-04.adoc` on valence-shell#230). The engine is the product and the shell interpreter is one front end, which comes last. What the interpreter does today is measured by `impl/rust-cli/tests/conformance_battery.rs` (66 of 126 command lines match bash), not by the list below, parts of which are out of date. +*Parked* by owner ruling OD-42 (2026-11-17; recorded in `docs/plans/VSH-ROUTE-PLAN-2026-11-04.adoc` on valence-shell#230). The engine is the product and the shell interpreter is one front end, which comes last. What the interpreter does today is measured by `impl/rust-cli/tests/conformance_battery.rs` (66 of 127 command lines match bash), not by the list below, parts of which are out of date. ==== Most critical missing POSIX features, ranked by impact (per CLAUDE.md diff --git a/impl/rust-cli/src/enhanced_repl.rs b/impl/rust-cli/src/enhanced_repl.rs index 3bf1f8e..f10ebb5 100644 --- a/impl/rust-cli/src/enhanced_repl.rs +++ b/impl/rust-cli/src/enhanced_repl.rs @@ -241,6 +241,8 @@ impl Prompt for VshPrompt { /// Run enhanced REPL pub fn run(state: &mut ShellState) -> Result<()> { + state.interactive = true; + // Set up history file let history_file = dirs::home_dir() .unwrap_or_else(|| PathBuf::from("/tmp")) @@ -406,6 +408,7 @@ fn execute_segments(state: &mut ShellState, input: &str) -> Result { Ok(false) => {} Err(e) => { eprintln!("{} {}", Color::Red.paint("Error:"), e); + state.last_exit_code = 1; // POSIX: continue executing remaining commands after error } } @@ -437,6 +440,50 @@ fn execute_line(state: &mut ShellState, input: &str) -> Result { state.last_exit_code = exit_code; Ok(false) } - ExecutionResult::Success => Ok(false), + ExecutionResult::Success => { + // A command that succeeded leaves status 0, as it does in a + // script. Without this `$?`, and the status `exit` hands on, + // stayed at whatever the last failing command left. + state.last_exit_code = 0; + Ok(false) + } + } +} + +#[cfg(test)] +mod status_tests { + use super::*; + + fn state() -> (tempfile::TempDir, ShellState) { + let root = tempfile::tempdir().expect("temporary directory"); + let state = ShellState::new(root.path().to_str().expect("a UTF-8 path")) + .expect("open the shell state"); + (root, state) + } + + /// A command that succeeds leaves status 0. It used to leave whatever + /// the last failing command had set, which `$?` and `exit` then showed. + #[test] + fn a_command_that_succeeds_resets_the_status() { + let (_root, mut state) = state(); + state.last_exit_code = 7; + + let leave = execute_line(&mut state, "mkdir made").expect("mkdir runs"); + + assert!(!leave); + assert_eq!(state.last_exit_code, 0); + } + + /// Typed at a prompt, `exit` with too many operands is reported and the + /// shell stays open, as in bash. + #[test] + fn exit_with_too_many_operands_keeps_an_interactive_shell_open() { + let (_root, mut state) = state(); + state.interactive = true; + + let leave = execute_line(&mut state, "exit 1 2").expect("exit is handled"); + + assert!(!leave); + assert_eq!(state.last_exit_code, 1); } } diff --git a/impl/rust-cli/src/executable.rs b/impl/rust-cli/src/executable.rs index 36b0584..04403c9 100644 --- a/impl/rust-cli/src/executable.rs +++ b/impl/rust-cli/src/executable.rs @@ -75,7 +75,7 @@ impl ExecutableCommand for Command { match self { // Filesystem operations (reversible) Command::Mkdir { path, redirects } => { - let expanded_path = crate::parser::expand_variables(path, state); + let expanded_path = expand_operand("mkdir", path, state)?; if redirects.is_empty() { commands::mkdir(state, &expanded_path, false)?; } else { @@ -87,7 +87,7 @@ impl ExecutableCommand for Command { } Command::Rmdir { path, redirects } => { - let expanded_path = crate::parser::expand_variables(path, state); + let expanded_path = expand_operand("rmdir", path, state)?; if redirects.is_empty() { commands::rmdir(state, &expanded_path, false)?; } else { @@ -99,7 +99,7 @@ impl ExecutableCommand for Command { } Command::Touch { path, redirects } => { - let expanded_path = crate::parser::expand_variables(path, state); + let expanded_path = expand_operand("touch", path, state)?; if redirects.is_empty() { commands::touch(state, &expanded_path, false)?; } else { @@ -111,7 +111,7 @@ impl ExecutableCommand for Command { } Command::Rm { path, redirects } => { - let expanded_path = crate::parser::expand_variables(path, state); + let expanded_path = expand_operand("rm", path, state)?; if redirects.is_empty() { commands::rm(state, &expanded_path, false)?; } else { @@ -127,7 +127,7 @@ impl ExecutableCommand for Command { force, redirects, } => { - let expanded_path = crate::parser::expand_variables(path, state); + let expanded_path = expand_operand("obliterate", path, state)?; if redirects.is_empty() { commands::secure_deletion::obliterate_path( state, @@ -148,8 +148,8 @@ impl ExecutableCommand for Command { dst, redirects, } => { - let expanded_src = crate::parser::expand_variables(src, state); - let expanded_dst = crate::parser::expand_variables(dst, state); + let expanded_src = expand_operand("cp", src, state)?; + let expanded_dst = expand_operand("cp", dst, state)?; if redirects.is_empty() { commands::cp(state, &expanded_src, &expanded_dst, false)?; } else { @@ -165,8 +165,8 @@ impl ExecutableCommand for Command { dst, redirects, } => { - let expanded_src = crate::parser::expand_variables(src, state); - let expanded_dst = crate::parser::expand_variables(dst, state); + let expanded_src = expand_operand("mv", src, state)?; + let expanded_dst = expand_operand("mv", dst, state)?; if redirects.is_empty() { commands::mv(state, &expanded_src, &expanded_dst, false)?; } else { @@ -182,8 +182,8 @@ impl ExecutableCommand for Command { link, redirects, } => { - let expanded_target = crate::parser::expand_variables(target, state); - let expanded_link = crate::parser::expand_variables(link, state); + let expanded_target = expand_operand("ln", target, state)?; + let expanded_link = expand_operand("ln", link, state)?; if redirects.is_empty() { commands::symlink(state, &expanded_target, &expanded_link, false)?; } else { @@ -199,8 +199,19 @@ impl ExecutableCommand for Command { path, redirects, } => { + // The mode may start with `-` (`chmod -x f`); only an option + // spelling that an expansion produced is refused. let expanded_mode = crate::parser::expand_variables(mode, state); - let expanded_path = crate::parser::expand_variables(path, state); + if crate::parser::is_chmod_option(&expanded_mode) + && !crate::parser::is_chmod_option(mode) + { + return Err(Refusal(format!( + "chmod: '{mode}' expands to '{expanded_mode}', an option the built-in \ + chmod does not handle; nothing was done" + )) + .into()); + } + let expanded_path = expand_operand("chmod", path, state)?; if redirects.is_empty() { commands::chmod(state, &expanded_mode, &expanded_path, false)?; } else { @@ -217,8 +228,8 @@ impl ExecutableCommand for Command { path, redirects, } => { - let expanded_owner = crate::parser::expand_variables(owner, state); - let expanded_path = crate::parser::expand_variables(path, state); + let expanded_owner = expand_operand("chown", owner, state)?; + let expanded_path = expand_operand("chown", path, state)?; if redirects.is_empty() { commands::chown(state, &expanded_owner, &expanded_path, false)?; } else { @@ -284,7 +295,19 @@ impl ExecutableCommand for Command { } // Exit commands - Command::Exit { code } => { + Command::Exit { code, too_many } => { + // More than one operand is an error. A shell that is running + // a script stops there with status 1, as bash and dash do. A + // shell a person is typing into reports it and stays open. + if *too_many { + eprintln!("vsh: exit: too many arguments"); + state.last_exit_code = 1; + return Ok(if state.interactive { + ExecutionResult::ExternalCommand { exit_code: 1 } + } else { + ExecutionResult::Exit + }); + } // `exit` alone keeps the status of the last command. Only the low // eight bits of a given status reach the caller (POSIX). if let Some(code) = code { @@ -306,7 +329,8 @@ impl ExecutableCommand for Command { // Expand variables in path let expanded_path = path .as_ref() - .map(|p| crate::parser::expand_variables(p, state)); + .map(|p| expand_operand("ls", p, state)) + .transpose()?; if redirects.is_empty() { // Direct output to terminal @@ -383,7 +407,8 @@ impl ExecutableCommand for Command { // Expand variables in path first let expanded_path = path .as_ref() - .map(|p| crate::parser::expand_variables(p, state)); + .map(|p| expand_operand("cd", p, state)) + .transpose()?; let target = if let Some(ref p) = expanded_path { if p == "-" { @@ -1476,6 +1501,44 @@ impl ExecutableCommand for Command { } } +/// Expand a built-in's operand, and refuse it if the expansion made it read +/// as an option. +/// +/// The parser has already refused the options a built-in does not handle, +/// but it sees the words as written. An operand written `$opt` passes it and +/// can still expand to `-p`. A program given that word would take it for an +/// option, so the built-in refuses it too, and does nothing, where it used +/// to make a directory named `-p`. +/// +/// A word that already read as an option when it was written can only have +/// come after `--`, which makes it an operand, so it is let through. +fn expand_operand(name: &str, raw: &str, state: &ShellState) -> Result { + let expanded = crate::parser::expand_variables(raw, state); + if crate::parser::reads_as_option(&expanded) && !crate::parser::reads_as_option(raw) { + return Err(Refusal(format!( + "{name}: '{raw}' expands to '{expanded}', which reads as an option the built-in \ + {name} does not handle; nothing was done (write ./{expanded} to name a file)" + )) + .into()); + } + Ok(expanded) +} + +/// A built-in refused its operands once they were expanded, and did nothing. +/// +/// A script treats this like a refusal found while the line was parsed: the +/// command fails with status 1 and the script goes on. +#[derive(Debug)] +pub struct Refusal(pub String); + +impl std::fmt::Display for Refusal { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +impl std::error::Error for Refusal {} + /// Execute a shell function call with arguments. /// /// Steps: diff --git a/impl/rust-cli/src/help.rs b/impl/rust-cli/src/help.rs index 4308965..a3def0b 100644 --- a/impl/rust-cli/src/help.rs +++ b/impl/rust-cli/src/help.rs @@ -147,12 +147,6 @@ impl HelpEntry { output.push('\n'); } - // Footer - output.push_str(&format!( - "Run {} for comprehensive documentation\n", - format!("man vsh-{}", self.name).bright_yellow() - )); - // Page if output is long pager::page(&output)?; Ok(()) diff --git a/impl/rust-cli/src/main.rs b/impl/rust-cli/src/main.rs index 3738fba..adf4898 100644 --- a/impl/rust-cli/src/main.rs +++ b/impl/rust-cli/src/main.rs @@ -282,7 +282,18 @@ fn execute_script_content(content: &str, state: &mut state::ShellState) -> Resul match vsh::parser::parse_command(segment) { Ok(cmd) => { - let result = cmd.execute(state)?; + let result = match cmd.execute(state) { + Ok(result) => result, + // A built-in that refused its operands has done nothing. + // Like a refusal found while parsing, it fails that + // command only. + Err(e) if e.is::() => { + eprintln!("vsh: {e}"); + state.last_exit_code = 1; + continue; + } + Err(e) => return Err(e), + }; match result { // `exit` has already set the status to leave with. Stop // here; the EXIT trap and the exit itself follow the loop. diff --git a/impl/rust-cli/src/pager.rs b/impl/rust-cli/src/pager.rs index 62bf050..d3d8c59 100644 --- a/impl/rust-cli/src/pager.rs +++ b/impl/rust-cli/src/pager.rs @@ -11,7 +11,7 @@ //! - Keyboard navigation (space/enter/q) use anyhow::{Context, Result}; -use std::io::{self, Write}; +use std::io::{self, IsTerminal, Write}; use std::process::{Command, Stdio}; /// Smart pager configuration @@ -192,7 +192,11 @@ impl Pager { pub fn page(&self, content: &str) -> Result<()> { let line_count = content.lines().count(); - if !self.should_page(line_count) { + // A pager is for a person at a terminal. When the output goes to a + // file or to another program, write it as it is: a pager there adds + // nothing, and one that does not take the options given to it (plain + // `more`, where there is no `less`) loses the text. + if !io::stdout().is_terminal() || !self.should_page(line_count) { // Output is short - just print directly print!("{}", content); io::stdout().flush()?; diff --git a/impl/rust-cli/src/parser.rs b/impl/rust-cli/src/parser.rs index a53e0bf..554e16d 100644 --- a/impl/rust-cli/src/parser.rs +++ b/impl/rust-cli/src/parser.rs @@ -259,8 +259,12 @@ pub enum Command { show_proofs: bool, }, /// `exit [status]`. The status is kept unexpanded until it runs. + /// `too_many` is set when more than one operand was given; what follows + /// then depends on whether a person is typing, so it is decided when the + /// command runs. Exit { code: Option, + too_many: bool, }, Quit, @@ -3282,7 +3286,7 @@ fn builtin_operands( for arg in args { if !options_ended && arg == "--" { options_ended = true; - } else if !options_ended && arg.len() > 1 && arg.starts_with('-') { + } else if !options_ended && reads_as_option(arg) { if !known_options.contains(&arg.as_str()) { return Err(anyhow!( "{name}: option '{arg}' is not supported by the built-in {name}; nothing was done" @@ -3310,6 +3314,18 @@ fn builtin_operands( Ok((options, operands)) } +/// Whether `word` reads as an option to a built-in: it starts with `-` and is +/// not `-` alone, which names standard input or the previous directory. +pub(crate) fn reads_as_option(word: &str) -> bool { + word.len() > 1 && word.starts_with('-') +} + +/// Whether `word`, where `chmod` expects its mode, is an option. A mode may +/// start with `-` (`chmod -x f`), so only the option spellings count. +pub(crate) fn is_chmod_option(word: &str) -> bool { + matches!(word, "-R" | "-c" | "-f" | "-v") || word.starts_with("--") +} + /// Parses the count given to `undo` or `redo`: nothing means one. fn step_count(name: &str, args: &[String]) -> Result { match args { @@ -3405,7 +3421,7 @@ fn parse_base_command( // The first argument is the mode, which may itself start with `-` // (`chmod -x f`), so only the option spellings are refused there. if let Some(first) = args.first() { - if matches!(first.as_str(), "-R" | "-c" | "-f" | "-v") || first.starts_with("--") { + if is_chmod_option(first) { return Err(anyhow!( "chmod: option '{first}' is not supported by the built-in chmod; nothing was done" )); @@ -3529,14 +3545,10 @@ fn parse_base_command( .unwrap_or(10); Ok(Command::History { count, show_proofs }) } - "exit" => { - if args.len() > 1 { - return Err(anyhow!("exit: too many arguments")); - } - Ok(Command::Exit { - code: args.first().cloned(), - }) - } + "exit" => Ok(Command::Exit { + code: args.first().cloned(), + too_many: args.len() > 1, + }), "quit" => Ok(Command::Quit), // Transactions diff --git a/impl/rust-cli/src/repl.rs b/impl/rust-cli/src/repl.rs index c861e84..018a42b 100644 --- a/impl/rust-cli/src/repl.rs +++ b/impl/rust-cli/src/repl.rs @@ -43,6 +43,8 @@ use crate::state::ShellState; /// # Ok::<(), anyhow::Error>(()) /// ``` pub fn run(state: &mut ShellState) -> Result<()> { + state.interactive = true; + // Install SIGINT handler for graceful Ctrl+C handling; a failure only // means Ctrl-C keeps its default behaviour, so warn and continue. if let Err(e) = ctrlc::set_handler(move || { @@ -90,6 +92,7 @@ pub fn run(state: &mut ShellState) -> Result<()> { } Err(e) => { eprintln!("{} {}", "Error:".bright_red(), e); + state.last_exit_code = 1; // POSIX: continue executing remaining commands after error } } @@ -180,7 +183,13 @@ fn execute_line(state: &mut ShellState, input: &str) -> Result { state.last_exit_code = exit_code; Ok(false) } - ExecutionResult::Success => Ok(false), + ExecutionResult::Success => { + // A command that succeeded leaves status 0, as it does in a + // script. Without this `$?`, and the status `exit` hands on, + // stayed at whatever the last failing command left. + state.last_exit_code = 0; + Ok(false) + } } } @@ -340,3 +349,41 @@ fn print_status(state: &ShellState) { state.transactions.len() ); } + +#[cfg(test)] +mod status_tests { + use super::*; + + fn state() -> (tempfile::TempDir, ShellState) { + let root = tempfile::tempdir().expect("temporary directory"); + let state = ShellState::new(root.path().to_str().expect("a UTF-8 path")) + .expect("open the shell state"); + (root, state) + } + + /// A command that succeeds leaves status 0. It used to leave whatever + /// the last failing command had set, which `$?` and `exit` then showed. + #[test] + fn a_command_that_succeeds_resets_the_status() { + let (_root, mut state) = state(); + state.last_exit_code = 7; + + let leave = execute_line(&mut state, "mkdir made").expect("mkdir runs"); + + assert!(!leave); + assert_eq!(state.last_exit_code, 0); + } + + /// Typed at a prompt, `exit` with too many operands is reported and the + /// shell stays open, as in bash. + #[test] + fn exit_with_too_many_operands_keeps_an_interactive_shell_open() { + let (_root, mut state) = state(); + state.interactive = true; + + let leave = execute_line(&mut state, "exit 1 2").expect("exit is handled"); + + assert!(!leave); + assert_eq!(state.last_exit_code, 1); + } +} diff --git a/impl/rust-cli/src/state.rs b/impl/rust-cli/src/state.rs index 7f75389..c48b1bf 100644 --- a/impl/rust-cli/src/state.rs +++ b/impl/rust-cli/src/state.rs @@ -284,6 +284,10 @@ pub struct ShellState { /// Last exit code from external command (for $? support) pub last_exit_code: i32, + /// Whether a person is typing into this shell. Set by the interactive + /// loops. A script, or `-c`, leaves it unset. + pub interactive: bool, + /// Previous directory for cd - support pub previous_dir: Option, @@ -346,6 +350,7 @@ impl ShellState { transactions: Vec::new(), state_file, last_exit_code: 0, + interactive: false, previous_dir: None, variables: HashMap::new(), exported_vars: HashSet::new(), diff --git a/impl/rust-cli/tests/builtin_strictness_tests.rs b/impl/rust-cli/tests/builtin_strictness_tests.rs index 982f900..3f2e555 100644 --- a/impl/rust-cli/tests/builtin_strictness_tests.rs +++ b/impl/rust-cli/tests/builtin_strictness_tests.rs @@ -218,6 +218,71 @@ fn a_double_dash_lets_a_name_start_with_a_dash() { assert!(dir.path().join("-p").is_dir()); } +/// The parser sees `$opt`, not what it holds. A word that reads as an option +/// only once it is expanded is refused as well: `opt=-p; mkdir $opt` used to +/// make a directory named `-p`. +#[test] +fn an_operand_that_expands_to_an_option_is_refused() { + refused("opt=-p; mkdir $opt", "expands to '-p'"); + refused("opt=-rf; rm $opt", "expands to '-rf'"); + refused("opt=-r; rmdir $opt", "expands to '-r'"); + refused("opt=-c; touch $opt", "expands to '-c'"); + refused("opt=-la; ls $opt", "expands to '-la'"); + refused("opt=-L; cd $opt", "expands to '-L'"); + refused("opt=-r; cp $opt a", "expands to '-r'"); + refused("opt=-f; mv a $opt", "expands to '-f'"); + refused("opt=-f; ln -s a $opt", "expands to '-f'"); + refused("opt=--force; obliterate $opt", "expands to '--force'"); + refused("opt=-R; chmod $opt a", "expands to '-R'"); + refused("opt=-v; chmod 600 $opt", "expands to '-v'"); +} + +/// Such a refusal fails its own command and no more, like one found while +/// the line was parsed. +#[test] +fn a_refusal_after_expansion_does_not_end_the_script() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh(dir.path(), "opt=-p; mkdir $opt; echo status=$?"); + + assert_eq!(run.stdout, "status=1\n", "stderr: {}", run.stderr); + assert!(tree(dir.path()).is_empty()); +} + +/// What an expansion may still produce: a mode that starts with a dash, and +/// the lone dash that means the previous directory. +#[test] +fn an_expansion_may_still_give_a_mode_or_a_lone_dash() { + let dir = fixture(); + + let run = vsh(dir.path(), "m=-w; chmod $m a; cd d; p=-; cd $p"); + + assert_eq!(run.code, 0, "stderr: {}", run.stderr); + let mode = fs::metadata(dir.path().join("a")) + .expect("metadata") + .permissions() + .mode(); + assert_eq!(mode & 0o222, 0, "write permission was removed"); +} + +/// `exit` with more than one operand is an error, and a script stops there +/// with status 1, as it does under bash and dash. It used to print the error +/// and carry on with the rest of the script. +#[test] +fn exit_with_too_many_operands_stops_the_script() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh(dir.path(), "exit 1 2; touch after"); + + assert_eq!(run.code, 1); + assert!( + run.stderr.contains("too many arguments"), + "stderr: {}", + run.stderr + ); + assert!(!dir.path().join("after").exists()); +} + #[test] fn a_refusal_does_not_end_the_script() { let dir = tempfile::tempdir().expect("temporary directory"); diff --git a/impl/rust-cli/tests/conformance/README.adoc b/impl/rust-cli/tests/conformance/README.adoc index 982d25c..c7a1820 100644 --- a/impl/rust-cli/tests/conformance/README.adoc +++ b/impl/rust-cli/tests/conformance/README.adoc @@ -61,9 +61,19 @@ Recorded from GNU bash 5.2.21 on 2026-11-17, each case in an empty directory. The five `rm-NN` cases were written by hand from the POSIX description of `rm`. Cases were chosen to give the same result on any Linux system: they create the -files they use and do not depend on what a system directory holds. Of the 126 +files they use and do not depend on what a system directory holds. Of the 127 cases, 21 use syntax only bash has; `dash` matches bash on the rest. +The test is built on Linux only. The expectations are from Linux, and several +cases use the GNU forms of `stat`, `wc` and other programs, which print +differently on other systems. + +One case was corrected after review. `rm-05` set up its two files with +`touch a b`, which the built-in refuses, so nothing was created, the `rm` had +nothing to remove, and the case matched for the wrong reason. It now creates +the files one at a time and fails, as it should: the built-in `rm` takes one +operand. A case must be able to fail: check that its set-up runs. + == Adding a case . Append a line to `battery.jsonl` with a new id of the form `group-NN`. @@ -76,4 +86,8 @@ cases, 21 use syntax only bash has; `dash` matches bash on the rest. * The battery is small and home-made. A public corpus should become the yardstick for any work on the shell language (review, section 11). * It runs only the non-interactive path (`vsh -c`). -* A case that does not finish within five seconds counts as failed. +* A case that does not finish within five seconds counts as failed, and the + shell and the commands in its process group are stopped. A command that has + moved to a process group of its own is not reached. +* A line of `battery.jsonl` that is not a whole case is an error, so a case + cannot drop out unseen. diff --git a/impl/rust-cli/tests/conformance/battery.jsonl b/impl/rust-cli/tests/conformance/battery.jsonl index 05dffb9..5ac631d 100644 --- a/impl/rust-cli/tests/conformance/battery.jsonl +++ b/impl/rust-cli/tests/conformance/battery.jsonl @@ -124,4 +124,5 @@ {"id": "rm-02", "rc": 0, "out": "rc=1\n", "cmd": "rm nosuch; echo rc=$?"} {"id": "rm-03", "rc": 0, "out": "rc=0\n", "cmd": "rm -f nosuch; echo rc=$?"} {"id": "rm-04", "rc": 0, "out": "0\n", "cmd": "mkdir d; touch d/x; rm -r d; ls | wc -l"} -{"id": "rm-05", "rc": 0, "out": "0\n", "cmd": "touch a b; rm a b; ls | wc -l"} +{"id": "rm-05", "rc": 0, "out": "0\n", "cmd": "touch a; touch b; rm a b; ls | wc -l"} +{"id": "builtin-20", "rc": 1, "out": "", "cmd": "exit 1 2; echo after"} diff --git a/impl/rust-cli/tests/conformance/known_failing.txt b/impl/rust-cli/tests/conformance/known_failing.txt index 90bb4c6..5c8a1e3 100644 --- a/impl/rust-cli/tests/conformance/known_failing.txt +++ b/impl/rust-cli/tests/conformance/known_failing.txt @@ -69,3 +69,4 @@ fs-10 # mkdir x y; ls | wc -l rm-02 # rm nosuch; echo rc=$? rm-03 # rm -f nosuch; echo rc=$? rm-04 # mkdir d; touch d/x; rm -r d; ls | wc -l +rm-05 # touch a; touch b; rm a b; ls | wc -l diff --git a/impl/rust-cli/tests/conformance_battery.rs b/impl/rust-cli/tests/conformance_battery.rs index fd53965..5e4d12b 100644 --- a/impl/rust-cli/tests/conformance_battery.rs +++ b/impl/rust-cli/tests/conformance_battery.rs @@ -25,10 +25,18 @@ //! `$CARGO_TARGET_TMPDIR/conformance-report.txt`. //! //! See `tests/conformance/README.adoc` for how to add a case. +//! +//! Linux only. The expectations were recorded on Linux, and several cases +//! use the GNU forms of `stat`, `wc` and others, which print differently +//! elsewhere. + +#![cfg(target_os = "linux")] use std::collections::{BTreeMap, BTreeSet}; use std::io::Read; +use std::os::unix::process::CommandExt; use std::process::{Command, Stdio}; +use std::sync::mpsc; use std::time::{Duration, Instant}; const BATTERY: &str = include_str!("conformance/battery.jsonl"); @@ -37,6 +45,10 @@ const KNOWN_FAILING: &str = include_str!("conformance/known_failing.txt"); /// Longest a single case may run. A case that does not finish counts as failed. const TIME_LIMIT: Duration = Duration::from_secs(5); +/// How long the output may stay open after the shell has finished. A command +/// the shell left running can hold it open. +const OUTPUT_GRACE: Duration = Duration::from_secs(2); + /// How many cases run at once. const WORKERS: usize = 8; @@ -55,38 +67,55 @@ struct Outcome { detail: String, } -fn cases() -> Vec { - BATTERY - .lines() - .filter(|line| !line.trim().is_empty()) - .filter_map(|line| { - let value: serde_json::Value = - serde_json::from_str(line).expect("battery.jsonl: every line is a JSON object"); - // The first line is a comment object carrying the licence header. - let id = value.get("id")?.as_str()?.to_string(); - let field = |name: &str| { +/// Reads the battery. The first line is a comment object that carries the +/// licence header; every other line is a case. A line that is not a whole +/// case is an error, so that a case cannot drop out of the battery unseen. +fn parse_battery(text: &str) -> Vec { + let mut lines = text.lines().filter(|line| !line.trim().is_empty()); + let header: serde_json::Value = + serde_json::from_str(lines.next().expect("battery.jsonl: a header line")) + .expect("battery.jsonl: the header line is a JSON object"); + assert!( + header.get("_comment").is_some() && header.get("id").is_none(), + "battery.jsonl: the first line is the comment object, not a case" + ); + lines + .enumerate() + .map(|(index, line)| { + let number = index + 2; + let value: serde_json::Value = serde_json::from_str(line).unwrap_or_else(|error| { + panic!("battery.jsonl line {number}: not a JSON object: {error}") + }); + let text_field = |name: &str| { value .get(name) - .unwrap_or_else(|| panic!("battery.jsonl: case {id} has no `{name}`")) - .clone() + .and_then(serde_json::Value::as_str) + .unwrap_or_else(|| { + panic!("battery.jsonl line {number}: `{name}` is missing or not a string") + }) + .to_string() }; - Some(Case { - rc: i32::try_from(field("rc").as_i64().expect("`rc` is a number")) - .expect("`rc` fits in i32"), - out: field("out") - .as_str() - .expect("`out` is a string") - .to_string(), - cmd: field("cmd") - .as_str() - .expect("`cmd` is a string") - .to_string(), - id, - }) + let rc = value + .get("rc") + .and_then(serde_json::Value::as_i64) + .and_then(|status| i32::try_from(status).ok()) + .unwrap_or_else(|| { + panic!("battery.jsonl line {number}: `rc` is missing or not a number") + }); + Case { + id: text_field("id"), + rc, + out: text_field("out"), + cmd: text_field("cmd"), + } }) .collect() } +fn cases() -> Vec { + parse_battery(BATTERY) +} + fn known_failing() -> BTreeSet { KNOWN_FAILING .lines() @@ -137,14 +166,33 @@ fn run(case: &Case) -> Outcome { .stdin(Stdio::null()) .stdout(Stdio::piped()) .stderr(Stdio::null()) + // Its own process group, so that a case which does not finish can be + // stopped together with the commands it started. + .process_group(0) .spawn() .expect("start the built shell"); + // Read the output while the shell runs. A case that writes more than a + // pipe holds would otherwise wait for a reader that starts only once the + // case has finished, and be counted as not finishing. + let stdout = child + .stdout + .take() + .expect("the shell's standard output is a pipe"); + let (sender, receiver) = mpsc::channel(); + std::thread::spawn(move || { + let mut stdout = stdout; + let mut raw = Vec::new(); + let _ = stdout.read_to_end(&mut raw); + let _ = sender.send(raw); + }); + let started = Instant::now(); let status = loop { match child.try_wait().expect("wait for the shell") { Some(status) => break Some(status), None if started.elapsed() >= TIME_LIMIT => { + stop_group(child.id()); let _ = child.kill(); let _ = child.wait(); break None; @@ -153,15 +201,18 @@ fn run(case: &Case) -> Outcome { } }; - let mut raw = Vec::new(); - if let Some(mut stdout) = child.stdout.take() { - let _ = stdout.read_to_end(&mut raw); - } - let got_out = normalise(&String::from_utf8_lossy(&raw)); + // The reader ends when every process that holds the pipe has gone. Do + // not wait for that without limit. + let raw = receiver.recv_timeout(OUTPUT_GRACE).ok(); - let (passed, detail) = match status { - None => (false, format!("did not finish within {TIME_LIMIT:?}")), - Some(status) => { + let (passed, detail) = match (status, raw) { + (None, _) => (false, format!("did not finish within {TIME_LIMIT:?}")), + (Some(_), None) => ( + false, + format!("its output was still open {OUTPUT_GRACE:?} after it finished"), + ), + (Some(status), Some(raw)) => { + let got_out = normalise(&String::from_utf8_lossy(&raw)); let got_rc = status.code().unwrap_or(-1); if got_out == case.out && got_rc == case.rc { (true, String::new()) @@ -171,7 +222,7 @@ fn run(case: &Case) -> Outcome { format!( "expected exit {} and {:?}; got exit {} and {:?}", case.rc, - case.out, + shorten(&case.out), got_rc, shorten(&got_out) ), @@ -186,6 +237,17 @@ fn run(case: &Case) -> Outcome { } } +/// Stops every process in the group led by `leader`. Best effort: a command +/// that has moved to a group of its own is not reached. +fn stop_group(leader: u32) { + let _ = Command::new("kill") + .args(["-KILL", "--", &format!("-{leader}")]) + .stdin(Stdio::null()) + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status(); +} + fn shorten(text: &str) -> String { const LIMIT: usize = 120; if text.chars().count() <= LIMIT { @@ -304,6 +366,63 @@ fn battery_matches_the_reference_except_for_the_known_failures() { ); } +/// A case line that lacks a field is an error. It used to be dropped, so a +/// mistyped `id` took its case out of the battery without a word. +#[test] +#[should_panic(expected = "line 2: `id` is missing")] +fn a_case_line_without_an_id_is_an_error() { + parse_battery("{\"_comment\": \"x\"}\n{\"rc\": 0, \"out\": \"\", \"cmd\": \"true\"}\n"); +} + +#[test] +#[should_panic(expected = "the first line is the comment object")] +fn the_first_line_must_be_the_comment_object() { + parse_battery("{\"id\": \"a-01\", \"rc\": 0, \"out\": \"\", \"cmd\": \"true\"}\n"); +} + +#[test] +fn every_line_after_the_first_is_a_case() { + let lines = BATTERY + .lines() + .filter(|line| !line.trim().is_empty()) + .count(); + assert_eq!(cases().len(), lines - 1); +} + +/// More output than a pipe holds (64 KiB on Linux) must not turn a case that +/// finishes into one that is counted as not finishing. +#[test] +fn output_larger_than_a_pipe_is_read_while_the_shell_runs() { + let out: String = (1..=30_000).map(|n| format!("{n}\n")).collect(); + assert!(out.len() > 2 * 65_536); + let outcome = run(&Case { + id: "big-output".to_string(), + rc: 0, + out, + cmd: "seq 1 30000".to_string(), + }); + assert!(outcome.passed, "{}", outcome.detail); +} + +/// A case that does not finish is stopped at the time limit and reported. +#[test] +fn a_case_that_does_not_finish_is_reported_within_the_limit() { + let started = Instant::now(); + let outcome = run(&Case { + id: "never-ends".to_string(), + rc: 0, + out: String::new(), + cmd: "sleep 600".to_string(), + }); + assert!(!outcome.passed); + assert!( + outcome.detail.contains("did not finish"), + "{}", + outcome.detail + ); + assert!(started.elapsed() < TIME_LIMIT + OUTPUT_GRACE + Duration::from_secs(5)); +} + #[test] fn receipt_lines_and_colour_codes_are_removed_before_comparing() { let raw = "\u{1b}[32mgreen\u{1b}[0m\n[op:1f163398] mkdir d\nkept\n[txn:558b5380] begin t\n"; diff --git a/impl/rust-cli/tests/help_tests.rs b/impl/rust-cli/tests/help_tests.rs index e3782f7..f82cbe0 100644 --- a/impl/rust-cli/tests/help_tests.rs +++ b/impl/rust-cli/tests/help_tests.rs @@ -59,8 +59,54 @@ fn help_for_an_unknown_command_says_so() { #[test] fn help_does_not_promise_things_that_are_not_there() { - let (_, stdout, _) = vsh("help"); + for script in ["help", "help mkdir", "help undo"] { + let (_, stdout, _) = vsh(script); - assert!(!stdout.contains("man vsh-"), "there are no manual pages"); - assert!(!stdout.contains("help -v"), "there is no -v option"); + assert!( + !stdout.contains("man vsh"), + "`{script}`: there are no manual pages" + ); + assert!( + !stdout.contains("help -v"), + "`{script}`: there is no -v option" + ); + } +} + +/// Help that is captured or piped is written as it is, not handed to a +/// pager. On a system with `more` and no `less` the pager was started with +/// options it does not take, and the help text was lost. +#[cfg(unix)] +#[test] +fn help_is_not_paged_when_it_is_not_going_to_a_terminal() { + use std::os::unix::fs::PermissionsExt; + + let dir = tempfile::tempdir().expect("temporary directory"); + let bin = tempfile::tempdir().expect("directory for a stand-in pager"); + // The only pager on the path, and it refuses whatever it is given. + let more = bin.path().join("more"); + std::fs::write(&more, "#!/bin/sh\nexit 1\n").expect("write the stand-in"); + std::fs::set_permissions(&more, std::fs::Permissions::from_mode(0o755)) + .expect("make it runnable"); + + let output = Command::new(env!("CARGO_BIN_EXE_vsh")) + .arg("-r") + .arg(dir.path()) + .arg("-c") + .arg("help mkdir") + .current_dir(dir.path()) + .env("PATH", bin.path()) + .env("NO_COLOR", "1") + .env("LINES", "5") + .env_remove("RUST_BACKTRACE") + .stdin(Stdio::null()) + .output() + .expect("start the built shell"); + let stdout = String::from_utf8_lossy(&output.stdout); + + assert!(output.status.success()); + assert!( + stdout.contains("mkdir") && stdout.lines().count() > 5, + "the help text should be on standard output: {stdout:?}" + ); } From 9038e78d815cd72430b2ea462f45ba510a84c0b7 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 15:25:30 +0000 Subject: [PATCH 07/14] fix(shell): a refusal is an ordinary failed command everywhere; exit and help details Four findings from the review of 0305e3a. Three are fixed here. - A refusal made after expansion was an error that only the top-level script loop caught. Inside a function, a loop or an `if` it ended the whole enclosing command: `f() { opt=-p; mkdir $opt; echo after; }; f` never reached `echo after`. A refusal is now an ordinary result with status 1, so whatever ran the command goes on, at any depth. The special case in the script loop is gone. - `exit nope 2` reported "too many arguments" and ended with status 1. Under bash and dash a status that is not a number comes first: status 2 (both run here). The checks are now in that order. - `help "$topic"` looked up the literal text. The topic is now expanded like any other operand. Not changed, and answered on its thread: `name=-p; mkdir -- "$name"` is still refused. The parser does not hand on where `--` stood, so the check after expansion cannot tell that word from an unprotected one. Carrying it through means changing how every built-in holds its operands. The refusal does nothing and says how to name the file, so it is safe; whether to do the larger change is the owner's call. One battery case added, `exit nope 2; echo after`, recorded from bash (dash agrees). The battery holds 128 cases and 67 match. Tests: two new in builtin_strictness_tests.rs, one in help_tests.rs. Against the code as pushed in 0305e3a all three fail. cargo fmt and clippy with -D warnings clean. cargo test --no-fail-fast: 862 passed, 0 failed, 15 ignored, with OLDPWD and PWD set and with both unset. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- CLAUDE.md | 4 +- README.adoc | 2 +- ROADMAP.adoc | 2 +- impl/rust-cli/src/executable.rs | 170 +++++++++++------- impl/rust-cli/src/main.rs | 13 +- .../tests/builtin_strictness_tests.rs | 39 ++++ impl/rust-cli/tests/conformance/README.adoc | 2 +- impl/rust-cli/tests/conformance/battery.jsonl | 1 + impl/rust-cli/tests/help_tests.rs | 9 + 9 files changed, 160 insertions(+), 82 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index b459347..2a10084 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -23,8 +23,8 @@ Opus audit session that fixed all broken tests, real bugs, and documentation. ### What Actually Works (v0.9.0) **Measured, not claimed (2026-11-17, gate G0).** `impl/rust-cli/tests/conformance_battery.rs` -starts the built shell on 127 everyday command lines and compares the result with bash: -**66 match**. The cases that fail are listed in +starts the built shell on 128 everyday command lines and compares the result with bash: +**67 match**. The cases that fail are listed in `impl/rust-cli/tests/conformance/known_failing.txt`. That list is a ratchet: it may only shrink, and a case may not be added to it to make a build pass. diff --git a/README.adoc b/README.adoc index 75b8e5b..be9e699 100644 --- a/README.adoc +++ b/README.adoc @@ -53,7 +53,7 @@ Valence Shell is a formally verified shell with proven reversibility guarantees. [NOTE] ==== -*Measured, not claimed.* `impl/rust-cli/tests/conformance_battery.rs` starts the built shell on 127 everyday command lines and compares the result with bash. On 2026-11-17, 66 match. The cases that fail are listed in `impl/rust-cli/tests/conformance/known_failing.txt`, which may only shrink. Below, `[x]` means it works through the built shell, `[~]` means in part, and the limit is stated. +*Measured, not claimed.* `impl/rust-cli/tests/conformance_battery.rs` starts the built shell on 128 everyday command lines and compares the result with bash. On 2026-11-17, 67 match. The cases that fail are listed in `impl/rust-cli/tests/conformance/known_failing.txt`, which may only shrink. Below, `[x]` means it works through the built shell, `[~]` means in part, and the limit is stated. ==== * [~] Built-in file commands with undo: `mkdir`, `rmdir`, `touch`, `rm`, `cp`, `mv`, `ln -s`, `chmod`, `chown`, in their plain forms only (one operand each, two for `cp`, `mv` and `ln -s`; no options). An option or an extra operand is refused and nothing is done diff --git a/ROADMAP.adoc b/ROADMAP.adoc index 67123c2..f7c3f98 100644 --- a/ROADMAP.adoc +++ b/ROADMAP.adoc @@ -88,7 +88,7 @@ integration), #56/#57/#58 (the three admits surfaced by #55). [IMPORTANT] ==== -*Parked* by owner ruling OD-42 (2026-11-17; recorded in `docs/plans/VSH-ROUTE-PLAN-2026-11-04.adoc` on valence-shell#230). The engine is the product and the shell interpreter is one front end, which comes last. What the interpreter does today is measured by `impl/rust-cli/tests/conformance_battery.rs` (66 of 127 command lines match bash), not by the list below, parts of which are out of date. +*Parked* by owner ruling OD-42 (2026-11-17; recorded in `docs/plans/VSH-ROUTE-PLAN-2026-11-04.adoc` on valence-shell#230). The engine is the product and the shell interpreter is one front end, which comes last. What the interpreter does today is measured by `impl/rust-cli/tests/conformance_battery.rs` (67 of 128 command lines match bash), not by the list below, parts of which are out of date. ==== Most critical missing POSIX features, ranked by impact (per CLAUDE.md diff --git a/impl/rust-cli/src/executable.rs b/impl/rust-cli/src/executable.rs index 04403c9..b5ddea0 100644 --- a/impl/rust-cli/src/executable.rs +++ b/impl/rust-cli/src/executable.rs @@ -75,7 +75,9 @@ impl ExecutableCommand for Command { match self { // Filesystem operations (reversible) Command::Mkdir { path, redirects } => { - let expanded_path = expand_operand("mkdir", path, state)?; + let Some(expanded_path) = expand_operand("mkdir", path, state) else { + return Ok(REFUSED); + }; if redirects.is_empty() { commands::mkdir(state, &expanded_path, false)?; } else { @@ -87,7 +89,9 @@ impl ExecutableCommand for Command { } Command::Rmdir { path, redirects } => { - let expanded_path = expand_operand("rmdir", path, state)?; + let Some(expanded_path) = expand_operand("rmdir", path, state) else { + return Ok(REFUSED); + }; if redirects.is_empty() { commands::rmdir(state, &expanded_path, false)?; } else { @@ -99,7 +103,9 @@ impl ExecutableCommand for Command { } Command::Touch { path, redirects } => { - let expanded_path = expand_operand("touch", path, state)?; + let Some(expanded_path) = expand_operand("touch", path, state) else { + return Ok(REFUSED); + }; if redirects.is_empty() { commands::touch(state, &expanded_path, false)?; } else { @@ -111,7 +117,9 @@ impl ExecutableCommand for Command { } Command::Rm { path, redirects } => { - let expanded_path = expand_operand("rm", path, state)?; + let Some(expanded_path) = expand_operand("rm", path, state) else { + return Ok(REFUSED); + }; if redirects.is_empty() { commands::rm(state, &expanded_path, false)?; } else { @@ -127,7 +135,9 @@ impl ExecutableCommand for Command { force, redirects, } => { - let expanded_path = expand_operand("obliterate", path, state)?; + let Some(expanded_path) = expand_operand("obliterate", path, state) else { + return Ok(REFUSED); + }; if redirects.is_empty() { commands::secure_deletion::obliterate_path( state, @@ -148,8 +158,12 @@ impl ExecutableCommand for Command { dst, redirects, } => { - let expanded_src = expand_operand("cp", src, state)?; - let expanded_dst = expand_operand("cp", dst, state)?; + let Some(expanded_src) = expand_operand("cp", src, state) else { + return Ok(REFUSED); + }; + let Some(expanded_dst) = expand_operand("cp", dst, state) else { + return Ok(REFUSED); + }; if redirects.is_empty() { commands::cp(state, &expanded_src, &expanded_dst, false)?; } else { @@ -165,8 +179,12 @@ impl ExecutableCommand for Command { dst, redirects, } => { - let expanded_src = expand_operand("mv", src, state)?; - let expanded_dst = expand_operand("mv", dst, state)?; + let Some(expanded_src) = expand_operand("mv", src, state) else { + return Ok(REFUSED); + }; + let Some(expanded_dst) = expand_operand("mv", dst, state) else { + return Ok(REFUSED); + }; if redirects.is_empty() { commands::mv(state, &expanded_src, &expanded_dst, false)?; } else { @@ -182,8 +200,12 @@ impl ExecutableCommand for Command { link, redirects, } => { - let expanded_target = expand_operand("ln", target, state)?; - let expanded_link = expand_operand("ln", link, state)?; + let Some(expanded_target) = expand_operand("ln", target, state) else { + return Ok(REFUSED); + }; + let Some(expanded_link) = expand_operand("ln", link, state) else { + return Ok(REFUSED); + }; if redirects.is_empty() { commands::symlink(state, &expanded_target, &expanded_link, false)?; } else { @@ -205,13 +227,15 @@ impl ExecutableCommand for Command { if crate::parser::is_chmod_option(&expanded_mode) && !crate::parser::is_chmod_option(mode) { - return Err(Refusal(format!( - "chmod: '{mode}' expands to '{expanded_mode}', an option the built-in \ - chmod does not handle; nothing was done" - )) - .into()); + eprintln!( + "vsh: chmod: '{mode}' expands to '{expanded_mode}', an option the \ + built-in chmod does not handle; nothing was done" + ); + return Ok(REFUSED); } - let expanded_path = expand_operand("chmod", path, state)?; + let Some(expanded_path) = expand_operand("chmod", path, state) else { + return Ok(REFUSED); + }; if redirects.is_empty() { commands::chmod(state, &expanded_mode, &expanded_path, false)?; } else { @@ -228,8 +252,12 @@ impl ExecutableCommand for Command { path, redirects, } => { - let expanded_owner = expand_operand("chown", owner, state)?; - let expanded_path = expand_operand("chown", path, state)?; + let Some(expanded_owner) = expand_operand("chown", owner, state) else { + return Ok(REFUSED); + }; + let Some(expanded_path) = expand_operand("chown", path, state) else { + return Ok(REFUSED); + }; if redirects.is_empty() { commands::chown(state, &expanded_owner, &expanded_path, false)?; } else { @@ -290,15 +318,35 @@ impl ExecutableCommand for Command { } Command::Help { topic } => { + let topic = topic + .as_ref() + .map(|t| crate::parser::expand_variables(t, state)); crate::help::display_help(topic.as_deref(), crate::help::HelpTier::Verbose)?; Ok(ExecutionResult::Success) } // Exit commands Command::Exit { code, too_many } => { + // A status that is not a number comes first: status 2, and + // the shell ends, whatever else was given (bash and dash). + let status = match code { + Some(code) => { + let text = crate::parser::expand_variables(code, state); + match text.trim().parse::() { + // Only the low eight bits reach the caller (POSIX). + Ok(status) => Some((status & 0xFF) as i32), + Err(_) => { + eprintln!("vsh: exit: {}: numeric argument required", text); + state.last_exit_code = 2; + return Ok(ExecutionResult::Exit); + } + } + } + None => None, + }; // More than one operand is an error. A shell that is running - // a script stops there with status 1, as bash and dash do. A - // shell a person is typing into reports it and stays open. + // a script stops there with status 1, as bash does. A shell a + // person is typing into reports it and stays open. if *too_many { eprintln!("vsh: exit: too many arguments"); state.last_exit_code = 1; @@ -308,17 +356,9 @@ impl ExecutableCommand for Command { ExecutionResult::Exit }); } - // `exit` alone keeps the status of the last command. Only the low - // eight bits of a given status reach the caller (POSIX). - if let Some(code) = code { - let text = crate::parser::expand_variables(code, state); - match text.trim().parse::() { - Ok(status) => state.last_exit_code = (status & 0xFF) as i32, - Err(_) => { - eprintln!("vsh: exit: {}: numeric argument required", text); - state.last_exit_code = 2; - } - } + // `exit` alone keeps the status of the last command. + if let Some(status) = status { + state.last_exit_code = status; } Ok(ExecutionResult::Exit) } @@ -327,10 +367,13 @@ impl ExecutableCommand for Command { // Navigation (built-ins but not reversible) Command::Ls { path, redirects } => { // Expand variables in path - let expanded_path = path - .as_ref() - .map(|p| expand_operand("ls", p, state)) - .transpose()?; + let expanded_path = match path { + Some(p) => match expand_operand("ls", p, state) { + Some(expanded) => Some(expanded), + None => return Ok(REFUSED), + }, + None => None, + }; if redirects.is_empty() { // Direct output to terminal @@ -405,10 +448,13 @@ impl ExecutableCommand for Command { use std::path::PathBuf; // Expand variables in path first - let expanded_path = path - .as_ref() - .map(|p| expand_operand("cd", p, state)) - .transpose()?; + let expanded_path = match path { + Some(p) => match expand_operand("cd", p, state) { + Some(expanded) => Some(expanded), + None => return Ok(REFUSED), + }, + None => None, + }; let target = if let Some(ref p) = expanded_path { if p == "-" { @@ -1501,8 +1547,14 @@ impl ExecutableCommand for Command { } } -/// Expand a built-in's operand, and refuse it if the expansion made it read -/// as an option. +/// What a built-in returns when it has refused its operands: it has done +/// nothing, and it fails with status 1. This is a result like any other +/// failed command's, so whatever ran the command goes on: a script, a +/// function body, a loop, or a person at the prompt. +const REFUSED: ExecutionResult = ExecutionResult::ExternalCommand { exit_code: 1 }; + +/// Expand a built-in's operand. If the expansion made it read as an option, +/// say so and return `None`; the caller then returns [`REFUSED`]. /// /// The parser has already refused the options a built-in does not handle, /// but it sees the words as written. An operand written `$opt` passes it and @@ -1511,34 +1563,22 @@ impl ExecutableCommand for Command { /// to make a directory named `-p`. /// /// A word that already read as an option when it was written can only have -/// come after `--`, which makes it an operand, so it is let through. -fn expand_operand(name: &str, raw: &str, state: &ShellState) -> Result { +/// come after `--`, which makes it an operand, so it is let through. A word +/// after `--` that becomes one only by expansion (`mkdir -- "$name"`) is +/// still refused: the parser does not hand on where `--` stood. The message +/// says how to name such a file. +fn expand_operand(name: &str, raw: &str, state: &ShellState) -> Option { let expanded = crate::parser::expand_variables(raw, state); if crate::parser::reads_as_option(&expanded) && !crate::parser::reads_as_option(raw) { - return Err(Refusal(format!( - "{name}: '{raw}' expands to '{expanded}', which reads as an option the built-in \ - {name} does not handle; nothing was done (write ./{expanded} to name a file)" - )) - .into()); - } - Ok(expanded) -} - -/// A built-in refused its operands once they were expanded, and did nothing. -/// -/// A script treats this like a refusal found while the line was parsed: the -/// command fails with status 1 and the script goes on. -#[derive(Debug)] -pub struct Refusal(pub String); - -impl std::fmt::Display for Refusal { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str(&self.0) + eprintln!( + "vsh: {name}: '{raw}' expands to '{expanded}', which reads as an option the \ + built-in {name} does not handle; nothing was done (write ./{expanded} to name a file)" + ); + return None; } + Some(expanded) } -impl std::error::Error for Refusal {} - /// Execute a shell function call with arguments. /// /// Steps: diff --git a/impl/rust-cli/src/main.rs b/impl/rust-cli/src/main.rs index adf4898..3738fba 100644 --- a/impl/rust-cli/src/main.rs +++ b/impl/rust-cli/src/main.rs @@ -282,18 +282,7 @@ fn execute_script_content(content: &str, state: &mut state::ShellState) -> Resul match vsh::parser::parse_command(segment) { Ok(cmd) => { - let result = match cmd.execute(state) { - Ok(result) => result, - // A built-in that refused its operands has done nothing. - // Like a refusal found while parsing, it fails that - // command only. - Err(e) if e.is::() => { - eprintln!("vsh: {e}"); - state.last_exit_code = 1; - continue; - } - Err(e) => return Err(e), - }; + let result = cmd.execute(state)?; match result { // `exit` has already set the status to leave with. Stop // here; the EXIT trap and the exit itself follow the loop. diff --git a/impl/rust-cli/tests/builtin_strictness_tests.rs b/impl/rust-cli/tests/builtin_strictness_tests.rs index 3f2e555..24fa0fa 100644 --- a/impl/rust-cli/tests/builtin_strictness_tests.rs +++ b/impl/rust-cli/tests/builtin_strictness_tests.rs @@ -249,6 +249,26 @@ fn a_refusal_after_expansion_does_not_end_the_script() { assert!(tree(dir.path()).is_empty()); } +/// The same inside a function, a loop and an `if`: the refused command fails +/// and the body goes on. A refusal used to be an error there, which ended +/// the whole enclosing command. +#[test] +fn a_refusal_after_expansion_does_not_end_a_function_a_loop_or_an_if() { + for script in [ + "f() { opt=-p; mkdir $opt; echo after; }; f", + "for x in 1; do opt=-p; mkdir $opt; echo after; done", + "if true; then opt=-p; mkdir $opt; echo after; fi", + ] { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh(dir.path(), script); + + assert_eq!(run.stdout, "after\n", "`{script}`: stderr {}", run.stderr); + assert!(run.stderr.contains("expands to '-p'"), "`{script}`"); + assert!(tree(dir.path()).is_empty(), "`{script}` made something"); + } +} + /// What an expansion may still produce: a mode that starts with a dash, and /// the lone dash that means the previous directory. #[test] @@ -283,6 +303,25 @@ fn exit_with_too_many_operands_stops_the_script() { assert!(!dir.path().join("after").exists()); } +/// A status that is not a number comes before the count of operands: status +/// 2, and the script stops, as under bash and dash. +#[test] +fn exit_with_a_status_that_is_not_a_number_ends_with_status_two() { + for script in ["exit nope; touch after", "exit nope 2; touch after"] { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh(dir.path(), script); + + assert_eq!(run.code, 2, "`{script}`"); + assert!( + run.stderr.contains("numeric argument required"), + "`{script}`: stderr {}", + run.stderr + ); + assert!(!dir.path().join("after").exists(), "`{script}`"); + } +} + #[test] fn a_refusal_does_not_end_the_script() { let dir = tempfile::tempdir().expect("temporary directory"); diff --git a/impl/rust-cli/tests/conformance/README.adoc b/impl/rust-cli/tests/conformance/README.adoc index c7a1820..49d1473 100644 --- a/impl/rust-cli/tests/conformance/README.adoc +++ b/impl/rust-cli/tests/conformance/README.adoc @@ -61,7 +61,7 @@ Recorded from GNU bash 5.2.21 on 2026-11-17, each case in an empty directory. The five `rm-NN` cases were written by hand from the POSIX description of `rm`. Cases were chosen to give the same result on any Linux system: they create the -files they use and do not depend on what a system directory holds. Of the 127 +files they use and do not depend on what a system directory holds. Of the 128 cases, 21 use syntax only bash has; `dash` matches bash on the rest. The test is built on Linux only. The expectations are from Linux, and several diff --git a/impl/rust-cli/tests/conformance/battery.jsonl b/impl/rust-cli/tests/conformance/battery.jsonl index 5ac631d..27dbd23 100644 --- a/impl/rust-cli/tests/conformance/battery.jsonl +++ b/impl/rust-cli/tests/conformance/battery.jsonl @@ -126,3 +126,4 @@ {"id": "rm-04", "rc": 0, "out": "0\n", "cmd": "mkdir d; touch d/x; rm -r d; ls | wc -l"} {"id": "rm-05", "rc": 0, "out": "0\n", "cmd": "touch a; touch b; rm a b; ls | wc -l"} {"id": "builtin-20", "rc": 1, "out": "", "cmd": "exit 1 2; echo after"} +{"id": "builtin-21", "rc": 2, "out": "", "cmd": "exit nope 2; echo after"} diff --git a/impl/rust-cli/tests/help_tests.rs b/impl/rust-cli/tests/help_tests.rs index f82cbe0..0851563 100644 --- a/impl/rust-cli/tests/help_tests.rs +++ b/impl/rust-cli/tests/help_tests.rs @@ -46,6 +46,15 @@ fn help_for_one_command_describes_it() { assert!(stdout.contains("mkdir"), "stdout: {stdout}"); } +/// The topic is expanded like any other operand. +#[test] +fn help_takes_its_topic_from_a_variable() { + let (code, stdout, stderr) = vsh("topic=mkdir; help $topic"); + + assert_eq!(code, 0, "stderr: {stderr}"); + assert!(stdout.contains("Create a directory"), "stdout: {stdout}"); +} + #[test] fn help_for_an_unknown_command_says_so() { let (code, _, stderr) = vsh("help nosuchcommand"); From 38e68b8d3a55edb426d98ed11eaba83f38d9eada Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 15:39:30 +0000 Subject: [PATCH 08/14] fix(shell): chmod takes its mode after `--`; help can be redirected Two findings from the review of 9038e78. Both are valid. - `chmod -- 600 file` and `chmod -- -x file` were refused: the `--` itself was taken for an option. `--` now ends the options for chmod as it does for the other built-ins. - `help mkdir > out` made no file and printed the text to the terminal: the redirection was parsed and then dropped. `help` now carries it, like the other commands that print. Tests: one in builtin_strictness_tests.rs, one in help_tests.rs. Both fail against the code as pushed in 9038e78. cargo fmt and clippy with -D warnings clean. cargo test --no-fail-fast: 864 passed, 0 failed, 15 ignored. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- impl/rust-cli/src/executable.rs | 12 +++++++-- impl/rust-cli/src/parser.rs | 25 ++++++++++++----- .../tests/builtin_strictness_tests.rs | 20 ++++++++++++++ impl/rust-cli/tests/help_tests.rs | 27 +++++++++++++++++++ 4 files changed, 76 insertions(+), 8 deletions(-) diff --git a/impl/rust-cli/src/executable.rs b/impl/rust-cli/src/executable.rs index b5ddea0..4bcb79d 100644 --- a/impl/rust-cli/src/executable.rs +++ b/impl/rust-cli/src/executable.rs @@ -317,11 +317,19 @@ impl ExecutableCommand for Command { Ok(ExecutionResult::Success) } - Command::Help { topic } => { + Command::Help { topic, redirects } => { let topic = topic .as_ref() .map(|t| crate::parser::expand_variables(t, state)); - crate::help::display_help(topic.as_deref(), crate::help::HelpTier::Verbose)?; + let show = + || crate::help::display_help(topic.as_deref(), crate::help::HelpTier::Verbose); + if redirects.is_empty() { + show()?; + } else { + // `help mkdir > file`: the redirection was parsed and + // then dropped, so the text went to the terminal. + redirection::capture_and_redirect(redirects, state, |_| show())?; + } Ok(ExecutionResult::Success) } diff --git a/impl/rust-cli/src/parser.rs b/impl/rust-cli/src/parser.rs index 554e16d..ea1290f 100644 --- a/impl/rust-cli/src/parser.rs +++ b/impl/rust-cli/src/parser.rs @@ -281,6 +281,7 @@ pub enum Command { /// `help [command]` Help { topic: Option, + redirects: Vec, }, Ls { path: Option, @@ -3418,21 +3419,32 @@ fn parse_base_command( }) } "chmod" => { + // `--` ends the options here as it does for the other built-ins: + // what follows is the mode and the file, whatever they look like. + let options_ended = args.first().is_some_and(|first| first == "--"); + let args = if options_ended { &args[1..] } else { &args[..] }; // The first argument is the mode, which may itself start with `-` // (`chmod -x f`), so only the option spellings are refused there. - if let Some(first) = args.first() { - if is_chmod_option(first) { - return Err(anyhow!( - "chmod: option '{first}' is not supported by the built-in chmod; nothing was done" - )); + if !options_ended { + if let Some(first) = args.first() { + if is_chmod_option(first) { + return Err(anyhow!( + "chmod: option '{first}' is not supported by the built-in chmod; nothing was done" + )); + } } } if args.len() < 2 { return Err(anyhow!("chmod: usage: chmod MODE FILE")); } + let mut after_mode: Vec = Vec::new(); + if options_ended { + after_mode.push("--".to_string()); + } + after_mode.extend_from_slice(&args[1..]); let (_, operands) = builtin_operands( "chmod", - &args[1..], + &after_mode, &[], 1, 1, @@ -3571,6 +3583,7 @@ fn parse_base_command( builtin_operands("help", &args, &[], 0, 1, "help: missing operand")?; Ok(Command::Help { topic: operands.first().cloned(), + redirects, }) } "ls" => { diff --git a/impl/rust-cli/tests/builtin_strictness_tests.rs b/impl/rust-cli/tests/builtin_strictness_tests.rs index 24fa0fa..cc57cd9 100644 --- a/impl/rust-cli/tests/builtin_strictness_tests.rs +++ b/impl/rust-cli/tests/builtin_strictness_tests.rs @@ -269,6 +269,26 @@ fn a_refusal_after_expansion_does_not_end_a_function_a_loop_or_an_if() { } } +/// `--` ends the options for `chmod` as for the other built-ins. It used to +/// be taken for an option itself and refused. +#[test] +fn chmod_takes_its_mode_after_a_double_dash() { + let dir = fixture(); + + let run = vsh(dir.path(), "chmod -- 600 a; chmod -- -w b"); + + assert_eq!(run.code, 0, "stderr: {}", run.stderr); + let mode = |name: &str| { + fs::metadata(dir.path().join(name)) + .expect("metadata") + .permissions() + .mode() + & 0o777 + }; + assert_eq!(mode("a"), 0o600); + assert_eq!(mode("b") & 0o222, 0, "write permission was removed"); +} + /// What an expansion may still produce: a mode that starts with a dash, and /// the lone dash that means the previous directory. #[test] diff --git a/impl/rust-cli/tests/help_tests.rs b/impl/rust-cli/tests/help_tests.rs index 0851563..9d51e16 100644 --- a/impl/rust-cli/tests/help_tests.rs +++ b/impl/rust-cli/tests/help_tests.rs @@ -55,6 +55,33 @@ fn help_takes_its_topic_from_a_variable() { assert!(stdout.contains("Create a directory"), "stdout: {stdout}"); } +/// `help` takes a redirection like any command that prints. The redirection +/// used to be parsed and dropped: no file was made and the text went to the +/// terminal. +#[test] +fn help_can_be_redirected_to_a_file() { + let dir = tempfile::tempdir().expect("temporary directory"); + let output = Command::new(env!("CARGO_BIN_EXE_vsh")) + .arg("-r") + .arg(dir.path()) + .arg("-c") + .arg("help mkdir > out") + .current_dir(dir.path()) + .env("NO_COLOR", "1") + .env_remove("RUST_BACKTRACE") + .stdin(Stdio::null()) + .output() + .expect("start the built shell"); + + assert!(output.status.success()); + let written = std::fs::read_to_string(dir.path().join("out")).expect("the file was made"); + assert!(written.contains("Create a directory"), "{written}"); + assert!( + !String::from_utf8_lossy(&output.stdout).contains("Create a directory"), + "the text should not also go to standard output" + ); +} + #[test] fn help_for_an_unknown_command_says_so() { let (code, _, stderr) = vsh("help nosuchcommand"); From 57068b459d198b70a0a98fb5d0e8f7a7d4ad132c Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 16:08:20 +0000 Subject: [PATCH 09/14] fix(shell): a redirection that does not take standard output leaves it alone `help mkdir 2>err` printed nothing, and so did `echo hi 2>err` and `echo hi Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- impl/rust-cli/src/redirection.rs | 9 +++++++++ impl/rust-cli/tests/help_tests.rs | 19 +++++++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/impl/rust-cli/src/redirection.rs b/impl/rust-cli/src/redirection.rs index 8c8c155..c624d60 100644 --- a/impl/rust-cli/src/redirection.rs +++ b/impl/rust-cli/src/redirection.rs @@ -58,6 +58,15 @@ where // Setup redirections (validates, opens files, records original state) let redirect_setup = RedirectSetup::setup(redirects, state)?; + // Only a redirection of standard output needs the output captured. + // With `2> file` or `< file` alone, capturing it threw it away: + // `echo hi 2>err` printed nothing. + if resolved_redirects.is_empty() { + f(state)?; + redirect_setup.record_for_undo(state)?; + return Ok(()); + } + // Capture stdout let mut buffer = gag::BufferRedirect::stdout()?; diff --git a/impl/rust-cli/tests/help_tests.rs b/impl/rust-cli/tests/help_tests.rs index 9d51e16..50a55c9 100644 --- a/impl/rust-cli/tests/help_tests.rs +++ b/impl/rust-cli/tests/help_tests.rs @@ -82,6 +82,25 @@ fn help_can_be_redirected_to_a_file() { ); } +/// `help mkdir 2>err` printed nothing: any redirection made the shell +/// capture standard output, and the captured text was written only to a +/// target that takes standard output. The same held for every built-in +/// that prints, so `echo` is checked here too. +#[test] +fn a_redirection_that_does_not_take_standard_output_leaves_it_alone() { + let (status, out, _) = vsh("help mkdir 2>err"); + assert_eq!(status, 0); + assert!(out.contains("Create a directory"), "{out}"); + + let (status, out, _) = vsh("echo hello 2>err"); + assert_eq!(status, 0); + assert_eq!(out, "hello\n"); + + let (status, out, _) = vsh("echo text > in; echo hello < in"); + assert_eq!(status, 0); + assert_eq!(out, "hello\n"); +} + #[test] fn help_for_an_unknown_command_says_so() { let (code, _, stderr) = vsh("help nosuchcommand"); From 9714586163cca11ca3e75dc977a93e7f74691234 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 16:36:42 +0000 Subject: [PATCH 10/14] fix(shell): a refused built-in fails its own command; the failing list is pinned; rmdir help A built-in that refused its arguments (an option it does not handle, too many operands, or none) was a parse error. A parse error stops the whole line from being read, so `mkdir -p x || echo fallback` printed nothing, an `if` whose condition was refused ran neither branch, and a loop whose body held a refusal did not run at all. The refusal is now a command, `Command::Refused`: it prints its message, does nothing, and fails with status 1, so `||`, `&&`, `if` and loops go on from the failure. `mkdir || echo fallback` (no operand) was wrong in the same way on `main` and is fixed with it. A line that is not well formed is still a parse error. The battery's list of known failures could grow without the test noticing: a case that had started to fail could be added to the file. The number of listed cases is now also written in the test. A case added to the file fails the test until the number is raised, and a case removed fails it until the number is lowered. A case taken off and another put on in one change is not caught; that needs a check against the base branch. `help rmdir` sent the reader to `rm -r`, which the built-in refuses. It now says there is no recursive removal. The one-line help form pointed to `help -v`, which is not an option; it points to `help `. Two existing tests asserted that a refusal is a parse error (`mkdir -p build`, and `obliterate` with no operand). They now assert that it is read as a refused command. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- impl/rust-cli/src/executable.rs | 8 +++ impl/rust-cli/src/help.rs | 5 +- impl/rust-cli/src/parser.rs | 58 ++++++++++++++++--- .../tests/builtin_strictness_tests.rs | 51 ++++++++++++++++ impl/rust-cli/tests/conformance/README.adoc | 10 +++- impl/rust-cli/tests/conformance_battery.rs | 30 ++++++++++ impl/rust-cli/tests/e2e_script_execution.rs | 6 +- impl/rust-cli/tests/help_tests.rs | 8 +++ impl/rust-cli/tests/obliterate_rmo_tests.rs | 12 +++- 9 files changed, 171 insertions(+), 17 deletions(-) diff --git a/impl/rust-cli/src/executable.rs b/impl/rust-cli/src/executable.rs index 4bcb79d..80685c1 100644 --- a/impl/rust-cli/src/executable.rs +++ b/impl/rust-cli/src/executable.rs @@ -333,6 +333,13 @@ impl ExecutableCommand for Command { Ok(ExecutionResult::Success) } + // A built-in that does not take what it was given: say so, + // do nothing, and fail like any other command that fails. + Command::Refused { message } => { + eprintln!("vsh: {message}"); + Ok(REFUSED) + } + // Exit commands Command::Exit { code, too_many } => { // A status that is not a number comes first: status 2, and @@ -1353,6 +1360,7 @@ impl ExecutableCommand for Command { Command::Graph => "graph".to_string(), Command::Proofs => "proofs".to_string(), Command::Help { .. } => "help".to_string(), + Command::Refused { .. } => "(refused)".to_string(), Command::Ls { path, .. } => { if let Some(p) = path { format!("ls {}", p) diff --git a/impl/rust-cli/src/help.rs b/impl/rust-cli/src/help.rs index a3def0b..ed77855 100644 --- a/impl/rust-cli/src/help.rs +++ b/impl/rust-cli/src/help.rs @@ -78,7 +78,7 @@ impl HelpEntry { println!("Usage: {}", self.usage.bright_cyan()); println!( "Run {} for details", - format!("help -v {}", self.name).bright_yellow() + format!("help {}", self.name).bright_yellow() ); Ok(()) } @@ -353,7 +353,8 @@ static HELP_DATABASE: &[HelpEntry] = &[ summary: "Remove an empty directory", usage: "rmdir ", description: "Removes an empty directory at the specified path.\n\ - Directory must be empty (use 'rm -r' for non-empty directories).\n\ + Directory must be empty. The built-in rm has no -r, so there is no\n\ + recursive removal: remove what is in the directory first.\n\ This operation is reversible via 'undo'.", examples: &[ Example { diff --git a/impl/rust-cli/src/parser.rs b/impl/rust-cli/src/parser.rs index ea1290f..68e92c5 100644 --- a/impl/rust-cli/src/parser.rs +++ b/impl/rust-cli/src/parser.rs @@ -283,6 +283,20 @@ pub enum Command { topic: Option, redirects: Vec, }, + + /// A built-in that was given arguments it does not take: an option it + /// does not handle, too many operands, or none where one is needed. + /// Running it prints `message` and fails with status 1; nothing else + /// is done. + /// + /// This is a command and not a parse error because the line is well + /// formed, and what follows depends on the failure: in + /// `mkdir -p x || echo fallback` the second command must run. As a + /// parse error it stopped the whole list, condition or loop from being + /// read at all. + Refused { + message: String, + }, Ls { path: Option, redirects: Vec, @@ -1865,10 +1879,33 @@ pub fn parse_command(input: &str) -> Result { let cmd = &command_tokens[0]; let args: Vec = command_tokens[1..].to_vec(); - // Parse base command with redirections - parse_base_command(cmd, args, redirects, background) + // Parse base command with redirections. A built-in that does not take + // what it was given is a command that fails when it is run (see + // `Command::Refused`). + match parse_base_command(cmd, args, redirects, background) { + Err(error) => match error.downcast::() { + Ok(Refusal(message)) => Ok(Command::Refused { message }), + Err(other) => Err(other), + }, + parsed => parsed, + } } +/// What a built-in says when it does not take the arguments it was given. +/// It is raised while the command is read, and becomes +/// [`Command::Refused`] there. Any other error from reading a command is +/// still a parse error. +#[derive(Debug)] +struct Refusal(String); + +impl std::fmt::Display for Refusal { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.0) + } +} + +impl std::error::Error for Refusal {} + /// Check if a string is a valid variable name /// Valid: starts with letter or underscore, then letters/digits/underscores fn is_valid_var_name(name: &str) -> bool { @@ -3289,9 +3326,10 @@ fn builtin_operands( options_ended = true; } else if !options_ended && reads_as_option(arg) { if !known_options.contains(&arg.as_str()) { - return Err(anyhow!( + return Err(Refusal(format!( "{name}: option '{arg}' is not supported by the built-in {name}; nothing was done" - )); + )) + .into()); } options.push(arg.clone()); } else { @@ -3299,7 +3337,7 @@ fn builtin_operands( } } if operands.len() < min { - return Err(anyhow!("{missing}")); + return Err(Refusal(missing.to_string()).into()); } if operands.len() > max { let takes = match max { @@ -3307,10 +3345,11 @@ fn builtin_operands( 1 => "one operand".to_string(), n => format!("{n} operands"), }; - return Err(anyhow!( + return Err(Refusal(format!( "{name}: the built-in {name} takes {takes}, but {} were given; nothing was done", operands.len() - )); + )) + .into()); } Ok((options, operands)) } @@ -3428,9 +3467,10 @@ fn parse_base_command( if !options_ended { if let Some(first) = args.first() { if is_chmod_option(first) { - return Err(anyhow!( + return Err(Refusal(format!( "chmod: option '{first}' is not supported by the built-in chmod; nothing was done" - )); + )) + .into()); } } } diff --git a/impl/rust-cli/tests/builtin_strictness_tests.rs b/impl/rust-cli/tests/builtin_strictness_tests.rs index cc57cd9..1baea9d 100644 --- a/impl/rust-cli/tests/builtin_strictness_tests.rs +++ b/impl/rust-cli/tests/builtin_strictness_tests.rs @@ -374,3 +374,54 @@ fn the_forms_the_built_ins_do_handle_still_work() { & 0o777; assert_eq!(mode, 0o600); } + +/// A built-in that refuses its arguments fails its own command, and what +/// follows goes on from that failure. The refusal used to be a parse +/// error, which stopped the whole list, condition or loop from being read: +/// `mkdir -p x || echo fallback` printed nothing at all. +#[test] +fn a_refused_built_in_fails_only_its_own_command() { + for (script, expected) in [ + ("mkdir -p x || echo fallback", "fallback\n"), + ("mkdir -p x && echo and; echo status=$?", "status=1\n"), + ("if mkdir -p x; then echo yes; else echo no; fi", "no\n"), + ( + "for i in 1 2; do mkdir -p x; echo after; done", + "after\nafter\n", + ), + ("mkdir a b || echo fallback", "fallback\n"), + ("mkdir || echo fallback", "fallback\n"), + ("chmod -R 600 f || echo fallback", "fallback\n"), + ("rm -r x; echo status=$?", "status=1\n"), + ] { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh(dir.path(), script); + + assert_eq!(run.stdout, expected, "`{script}`: {}", run.stderr); + assert!( + tree(dir.path()).is_empty(), + "`{script}` changed the directory: {:?}", + tree(dir.path()) + ); + } + + // On its own, the refusal is the status of the line. + let dir = tempfile::tempdir().expect("temporary directory"); + let run = vsh(dir.path(), "mkdir -p x"); + assert_eq!(run.code, 1, "{}", run.stderr); + assert!(run.stderr.contains("not supported"), "{}", run.stderr); +} + +/// A line that is not well formed is still a parse error, not a refusal. +#[test] +fn a_malformed_line_is_still_a_parse_error() { + use vsh::parser::{parse_command, Command as Parsed}; + + assert!(parse_command("if [ true then echo yes fi").is_err()); + assert!(parse_command("echo \"unclosed").is_err()); + assert!(matches!( + parse_command("mkdir -p x"), + Ok(Parsed::Refused { .. }) + )); +} diff --git a/impl/rust-cli/tests/conformance/README.adoc b/impl/rust-cli/tests/conformance/README.adoc index 49d1473..d55c637 100644 --- a/impl/rust-cli/tests/conformance/README.adoc +++ b/impl/rust-cli/tests/conformance/README.adoc @@ -41,6 +41,13 @@ A report of every case is written to `target/tmp/conformance-report.txt`. until it is. * Do not add a case to the list to make a failing build pass. Fix the shell. If an expectation is wrong, correct the case and say why in the commit. +* The number of cases on the list is also written in the test + (`KNOWN_FAILING_COUNT`). A case added to the file fails the test until that + number is raised, and a case removed fails it until the number is lowered. + So the list cannot grow by an edit to the list alone. One thing is not + caught: a case taken off and another put on in the same change. A check + against the base branch would catch that; it needs a workflow, and the Rust + workflows do not start at present. == What is compared @@ -79,7 +86,8 @@ operand. A case must be able to fail: check that its set-up runs. . Append a line to `battery.jsonl` with a new id of the form `group-NN`. . Take `out` and `rc` from a reference shell run in an empty directory. . Run the test. If the case fails, the shell has a gap: add the id to - `known_failing.txt` in the same change and say so. + `known_failing.txt` and raise `KNOWN_FAILING_COUNT` in the same change, and + say so. == Limits diff --git a/impl/rust-cli/tests/conformance_battery.rs b/impl/rust-cli/tests/conformance_battery.rs index 5e4d12b..04482f4 100644 --- a/impl/rust-cli/tests/conformance_battery.rs +++ b/impl/rust-cli/tests/conformance_battery.rs @@ -42,6 +42,20 @@ use std::time::{Duration, Instant}; const BATTERY: &str = include_str!("conformance/battery.jsonl"); const KNOWN_FAILING: &str = include_str!("conformance/known_failing.txt"); +/// How many cases the list of known failures holds. It may only go down. +/// +/// The list is a file. A test that only read the file could be satisfied +/// by adding a case that has started to fail. With the number here, in +/// the test, a case added to the file fails the test until this number is +/// raised too, and raising it is a change to the gate itself, not to a +/// list. When a case is fixed and leaves the file, lower the number to +/// match: the test fails until it is lowered, so it cannot drift back up. +/// +/// What this does not catch: one case taken off the list and another put +/// on in the same change. Only a comparison with the base branch catches +/// that. +const KNOWN_FAILING_COUNT: usize = 61; + /// Longest a single case may run. A case that does not finish counts as failed. const TIME_LIMIT: Duration = Duration::from_secs(5); @@ -429,6 +443,22 @@ fn receipt_lines_and_colour_codes_are_removed_before_comparing() { assert_eq!(normalise(raw), "green\nkept\n"); } +/// See [`KNOWN_FAILING_COUNT`]. +#[test] +fn the_known_failing_list_has_not_grown() { + let listed = known_failing().len(); + assert!( + listed <= KNOWN_FAILING_COUNT, + "known_failing.txt holds {listed} cases and the gate allows {KNOWN_FAILING_COUNT}. \ + A case that has started to fail is a regression: fix it, do not list it." + ); + assert_eq!( + listed, KNOWN_FAILING_COUNT, + "known_failing.txt has shrunk to {listed} cases. Lower KNOWN_FAILING_COUNT to \ + {listed}, so that the list cannot grow back." + ); +} + #[test] fn the_known_failing_list_ignores_comments_and_blank_lines() { let parsed: BTreeSet = "# heading\n\nlogic-03 # bracket test\n \nloop-01\n" diff --git a/impl/rust-cli/tests/e2e_script_execution.rs b/impl/rust-cli/tests/e2e_script_execution.rs index 34e177c..ed0be85 100644 --- a/impl/rust-cli/tests/e2e_script_execution.rs +++ b/impl/rust-cli/tests/e2e_script_execution.rs @@ -14,7 +14,7 @@ mod e2e_script_execution { use std::fs; use tempfile::TempDir; use vsh::commands::mkdir; - use vsh::parser::parse_command; + use vsh::parser::{parse_command, Command}; use vsh::state::ShellState; // ================================================================ @@ -302,8 +302,10 @@ mkdir test_dir // The built-in mkdir has no `-p`. It used to take `-p` for the name of // the directory; it now refuses the option (see // `builtin_strictness_tests.rs`). + // The refusal is a command that fails when it is run, so that + // `mkdir -p build || ...` can go on to its second command. let result = parse_command("mkdir -p build"); - assert!(result.is_err()); + assert!(matches!(result, Ok(Command::Refused { .. })), "{result:?}"); // Without the option it parses and runs. assert!(parse_command("mkdir build").is_ok()); diff --git a/impl/rust-cli/tests/help_tests.rs b/impl/rust-cli/tests/help_tests.rs index 50a55c9..470646d 100644 --- a/impl/rust-cli/tests/help_tests.rs +++ b/impl/rust-cli/tests/help_tests.rs @@ -126,6 +126,14 @@ fn help_does_not_promise_things_that_are_not_there() { "`{script}`: there is no -v option" ); } + + // `help rmdir` sent the reader to `rm -r`, which the built-in refuses. + let (_, stdout, _) = vsh("help rmdir"); + assert!(stdout.contains("no -r"), "{stdout}"); + assert!( + !stdout.contains("use 'rm -r'"), + "`help rmdir` recommends a form that is refused" + ); } /// Help that is captured or piped is written as it is, not handed to a diff --git a/impl/rust-cli/tests/obliterate_rmo_tests.rs b/impl/rust-cli/tests/obliterate_rmo_tests.rs index 64f9e32..7fa63fc 100644 --- a/impl/rust-cli/tests/obliterate_rmo_tests.rs +++ b/impl/rust-cli/tests/obliterate_rmo_tests.rs @@ -47,11 +47,17 @@ fn obliterate_is_wired_into_dispatch() { } } -/// obliterate with no operand is a clean error, not a panic. +/// obliterate with no operand is a clean refusal, not a panic. It is read +/// as a command that fails when it is run, and it obliterates nothing. #[test] fn obliterate_requires_operand() { - assert!(parse_command("obliterate").is_err()); - assert!(parse_command("obliterate --force").is_err()); + for line in ["obliterate", "obliterate --force"] { + let parsed = parse_command(line); + assert!( + matches!(parsed, Ok(vsh::parser::Command::Refused { .. })), + "{line}: {parsed:?}" + ); + } } /// End-to-end: parse + execute actually destroys the file. From ec6aa14e7a2c93187a5c4530202348b5e7abdb84 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 00:00:35 +0000 Subject: [PATCH 11/14] docs(shell): the module comment of the binary says what the shell is The comment at the top of `main.rs` called the shell "formally verified", promised undo and redo for all filesystem operations, and said that complex operations go to a BEAM daemon and that a Zig FFI layer provides a verified POSIX interface. None of the four holds: the proofs are about a model, `obliterate` cannot be undone and redo is not implemented for every operation, the daemon is not built, and the Zig layer is not connected to this shell. The comment now says so, as the `proofs` command already does. No behaviour changes: the comment is not printed anywhere. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- impl/rust-cli/src/main.rs | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/impl/rust-cli/src/main.rs b/impl/rust-cli/src/main.rs index 3738fba..7f8f99e 100644 --- a/impl/rust-cli/src/main.rs +++ b/impl/rust-cli/src/main.rs @@ -1,18 +1,21 @@ // SPDX-License-Identifier: MPL-2.0 // Copyright (c) Jonathan D.A. Jewell -//! Valence Shell (vsh) - A Formally Verified Reversible Shell +//! Valence Shell (vsh) - a reversible shell, a research prototype //! //! This is the user-facing CLI for Valence Shell. It provides: //! - Interactive shell with command history -//! - Undo/redo for all filesystem operations +//! - Undo, and redo where it is implemented, for the shell's own +//! filesystem operations (`obliterate` cannot be undone) //! - Transaction grouping //! - Proof references for each operation //! - Shell script execution (`vsh script.sh`, `vsh -c "cmd"`) //! //! Architecture: -//! - This Rust binary handles UI and fast-path operations -//! - Complex operations delegate to BEAM daemon -//! - Zig FFI provides the verified POSIX interface +//! - This Rust binary does everything the shell does today +//! - The proofs are about a model; the link from the model to this code is +//! tested, not proved (see the `proofs` command) +//! - The BEAM daemon is planned and not built, and the Zig FFI layer is not +//! connected to this shell //! //! Author: Jonathan D.A. Jewell From 8e20abc8aecdba1d73f772ee7407876da0006809 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 00:36:24 +0000 Subject: [PATCH 12/14] fix(shell): status after a list or a condition; EXIT trap; replay says what its theorems claim Sixth review round on the baseline. Status and control flow - `$?` is now set after the left side of `&&` and `||`, and after the condition of `if`, `elif`, `while` and `until`. The right side, the branch or the body sees the status of what ran just before it. It saw the status of an earlier command. - A function whose last line cannot be read fails with status 1. It kept the result of the line before. - `undo` and `redo` with a count that cannot be read are refused, so they fail their own command only: `undo many || echo no` prints `no`. - An EXIT trap runs every statement of its text. It was read as one command, so `trap 'echo a; echo b' EXIT` ran only the first. Replay and the proof references - A replay says what the theorem shown for each step claims, and its closing line no longer says that each operation has a reversal theorem. A redirection is shown under the theorem for a write of a whole file. A change to a variable has no theorem. An obliteration and a hardware erase have a theorem that the operation is not injective in the model. - The composition theorem is shown with what it covers, and the undo path does not end at the first state when a step cannot be reversed. - Three names the shell showed were declared in no proof file: `obliterate_irreversible` (the theorem is `obliterate_not_injective`, in Coq and Lean), `hardware_erase_irreversible` (the statement is `hardwareEraseIrreversible`, in the Idris2 layer only) and `variable_assignment_reversible` (no theorem; the three files the entry named do not exist). The entries now say so. - A unit test reads the proof files and checks that each name shown is declared in a file its entry names. It checks names only. The line numbers in the entries are not checked, and several are out of date; that and the names per prover belong to the manifest work (route plan E5). Texts and the battery - Help, CLAUDE.md and README give the operand counts of `chmod` and `chown` (a mode or an owner, then one file) and say which failures end a `-c` script. - The battery gives each child a HOME, so a runner without one does not report a difference in `vars-09`. - The battery README no longer tells anyone to list a new failing case: the list of known failures only shrinks. Checked: 879 tests pass and 15 are ignored, with and without OLDPWD and PWD; clippy and fmt are clean. Eighteen single mutations of the new code each fail a test. The Idris2 file was read, not compiled: Idris2 is not installed here. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- CLAUDE.md | 10 +- README.adoc | 2 +- impl/rust-cli/src/commands.rs | 233 ++++++++++++++- impl/rust-cli/src/executable.rs | 12 + impl/rust-cli/src/help.rs | 2 +- impl/rust-cli/src/main.rs | 53 ++-- impl/rust-cli/src/parser.rs | 10 +- impl/rust-cli/src/proof_refs.rs | 266 +++++++++++++++--- .../tests/builtin_strictness_tests.rs | 57 ++++ impl/rust-cli/tests/conformance/README.adoc | 14 +- impl/rust-cli/tests/conformance_battery.rs | 3 + .../tests/function_control_flow_tests.rs | 33 +++ 12 files changed, 620 insertions(+), 75 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 2a10084..dd84010 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -31,8 +31,10 @@ shrink, and a case may not be added to it to make a build pass. Works through the built shell (passing battery cases, or checked by hand interactively): - Built-in file operations with undo/redo (`mkdir`, `rmdir`, `touch`, `rm`, `cp`, `mv`, - `ln -s`, `chmod`, `chown`) in their plain forms only: one operand each (two for `cp`, - `mv`, `ln -s`) and no options. An option or an extra operand is refused, and nothing is done + `ln -s`, `chmod`, `chown`) in their plain forms only: one operand each, two for `cp`, + `mv` and `ln -s`, and a mode or an owner followed by one file for `chmod` and `chown`. None + takes an option (the `-s` of `ln -s` apart). An option or an extra operand is refused, and + nothing is done - External command execution via PATH lookup; pipelines of external programs - `>`, `>>`, `<` and here strings (`<</dev/null`) -- A failing built-in ends a `-c` script +- A built-in that fails with an error (`mkdir` on a directory that exists, say) ends a `-c` + script, and the rest of an `&&` or `||` list with it. A refused built-in and `false` do + not: they fail with status 1 and the script goes on - Ctrl-Z does not suspend a running program Not exercised by the battery: process substitution, command correction, the pager. diff --git a/README.adoc b/README.adoc index be9e699..2269cfc 100644 --- a/README.adoc +++ b/README.adoc @@ -56,7 +56,7 @@ Valence Shell is a formally verified shell with proven reversibility guarantees. *Measured, not claimed.* `impl/rust-cli/tests/conformance_battery.rs` starts the built shell on 128 everyday command lines and compares the result with bash. On 2026-11-17, 67 match. The cases that fail are listed in `impl/rust-cli/tests/conformance/known_failing.txt`, which may only shrink. Below, `[x]` means it works through the built shell, `[~]` means in part, and the limit is stated. ==== -* [~] Built-in file commands with undo: `mkdir`, `rmdir`, `touch`, `rm`, `cp`, `mv`, `ln -s`, `chmod`, `chown`, in their plain forms only (one operand each, two for `cp`, `mv` and `ln -s`; no options). An option or an extra operand is refused and nothing is done +* [~] Built-in file commands with undo: `mkdir`, `rmdir`, `touch`, `rm`, `cp`, `mv`, `ln -s`, `chmod`, `chown`, in their plain forms only (one operand each; two for `cp`, `mv` and `ln -s`; a mode or an owner and then one file for `chmod` and `chown`; no options, the `-s` of `ln -s` apart). An option or an extra operand is refused and nothing is done * [ ] Wildcard, brace and tilde expansion: applied only when the command also has a redirection * [ ] Quoting: a quoted `*`, `?`, `[`, `{` or `~` reaches other programs with a backslash added * [x] Unix Pipelines: Multi-stage pipelines of external programs (`cmd1 | cmd2 | cmd3`) diff --git a/impl/rust-cli/src/commands.rs b/impl/rust-cli/src/commands.rs index fa7e538..b8cc3ad 100644 --- a/impl/rust-cli/src/commands.rs +++ b/impl/rust-cli/src/commands.rs @@ -2524,6 +2524,102 @@ pub fn diff_state(state: &ShellState, target_op: usize) -> Result<()> { Ok(()) } +/// What a replay says at the end about the theorems it showed. +/// +/// It used to say that each operation has a reversal theorem. That is not +/// so for an obliteration or a hardware erase, whose theorem says that the +/// operation is not injective, nor for a redirection, which is shown under +/// the theorem for a write of a whole file, nor for a change to a variable, +/// for which no proof file states a theorem. +fn replay_summary(claims: &[crate::proof_refs::Claim]) -> String { + use crate::proof_refs::Claim; + + let count = |wanted: Claim| claims.iter().filter(|claim| **claim == wanted).count(); + let link = "the link from the model to this code is tested, not proved."; + let reversal = count(Claim::Reversal); + if reversal == claims.len() { + return format!("Each has a reversal theorem about the model; {link}"); + } + let mut parts = vec![format!( + "With a reversal theorem about the model: {reversal}." + )]; + let others = [ + ( + Claim::ReversalOfAWrite, + "Redirections, shown under the theorem for a write of a whole file \ + (none is stated for a redirection)", + ), + ( + Claim::ReversalNotStated, + "Reversed from the shell's own record, with no theorem stated", + ), + ( + Claim::Irreversible, + "Not reversible, with a theorem that the operation is not injective in the model", + ), + ( + Claim::IrreversibleNotStated, + "Not reversible, with no theorem stated", + ), + ]; + for (claim, text) in others { + let n = count(claim); + if n > 0 { + parts.push(format!("{text}: {n}.")); + } + } + parts.push(format!("Where a theorem is shown, {link}")); + parts.join(" ") +} + +/// What a replay adds after the theorem it shows for one step. +fn replay_note(claim: crate::proof_refs::Claim) -> &'static str { + use crate::proof_refs::Claim; + + match claim { + Claim::Reversal => "", + Claim::ReversalOfAWrite => " (stated for a write of a whole file, not for a redirection)", + Claim::ReversalNotStated => " (the shell reverses this from its own record)", + Claim::Irreversible => " (not injective in the model: this operation cannot be reversed)", + Claim::IrreversibleNotStated => " (this operation cannot be reversed)", + } +} + +/// What a replay adds after the composition theorem. That theorem is about +/// a sequence of operations that each have a reversal theorem, so it does +/// not cover a replay that holds any other kind. +fn replay_composition_note(claims: &[crate::proof_refs::Claim]) -> String { + use crate::proof_refs::Claim; + + let covered = claims + .iter() + .filter(|claim| **claim == Claim::Reversal) + .count(); + if covered == claims.len() { + return String::new(); + } + format!( + " (about operations that each have a reversal theorem: {covered} of the {} here)", + claims.len() + ) +} + +/// The last line of the undo path of a replay. The path ends at the first +/// state only if every operation in it can be reversed. +fn replay_path_end(claims: &[crate::proof_refs::Claim]) -> String { + use crate::proof_refs::Claim; + + let irreversible = claims + .iter() + .filter(|claim| matches!(claim, Claim::Irreversible | Claim::IrreversibleNotStated)) + .count(); + if irreversible == 0 { + " → fs₀".to_string() + } else { + format!(" → not fs₀: {irreversible} of these cannot be reversed") + } +} + /// Replay: animated replay of operation history with proof narration. pub fn replay(state: &ShellState, start: usize, end: usize) -> Result<()> { let end = end.min(state.history.len()); @@ -2565,7 +2661,12 @@ pub fn replay(state: &ShellState, start: usize, end: usize) -> Result<()> { format!("_{}", idx) }; println!(" State: fs{} ──▶ fs_{}", from_state, idx + 1); - println!(" Theorem (model): {}", proof.theorem.bright_cyan()); + let note = replay_note(crate::proof_refs::ProofReference::claim_for(op.op_type)); + println!( + " Theorem (model): {}{}", + proof.theorem.bright_cyan(), + note + ); println!(" [{}] {}%", bar.bright_green(), (step * 100) / total); println!(); @@ -2578,20 +2679,22 @@ pub fn replay(state: &ShellState, start: usize, end: usize) -> Result<()> { "Total".bright_white().bold(), total ); - println!( - "Each has a reversal theorem about the model; the link from the model to this code is tested, not proved." - ); + let claims: Vec = (start..end) + .map(|idx| crate::proof_refs::ProofReference::claim_for(state.history[idx].op_type)) + .collect(); + println!("{}", replay_summary(&claims)); let composition = crate::proof_refs::COMPOSITION_REVERSIBLE; println!( - "Composition theorem (model): {}", - composition.theorem.bright_cyan() + "Composition theorem (model): {}{}", + composition.theorem.bright_cyan(), + replay_composition_note(&claims) ); - // Show full undo path + // Show the undo path if total > 0 { println!(); - println!("{}", "Full undo path:".bright_yellow()); + println!("{}", "Undo path:".bright_yellow()); for idx in (start..end).rev() { let op = &state.history[idx]; let inv = op @@ -2604,7 +2707,7 @@ pub fn replay(state: &ShellState, start: usize, end: usize) -> Result<()> { } println!(); } - println!(" → fs₀"); + println!("{}", replay_path_end(&claims)); } Ok(()) @@ -2614,6 +2717,118 @@ pub fn replay(state: &ShellState, start: usize, end: usize) -> Result<()> { mod tests { use super::*; + /// A replay that holds an operation with no inverse does not say that + /// its undo path ends at the first state, nor that the composition + /// theorem covers it. It said both. + #[test] + fn test_replay_does_not_claim_a_way_back_past_an_irreversible_step() { + use crate::proof_refs::Claim; + + let reversible = [Claim::Reversal, Claim::Reversal]; + assert_eq!(replay_path_end(&reversible), " → fs₀"); + assert_eq!(replay_composition_note(&reversible), ""); + + let with_obliteration = [Claim::Reversal, Claim::Irreversible]; + let end = replay_path_end(&with_obliteration); + assert!(end.contains("not fs₀"), "{end}"); + assert!(end.contains("1 of these cannot be reversed"), "{end}"); + let note = replay_composition_note(&with_obliteration); + assert!(note.contains("1 of the 2 here"), "{note}"); + + // A change to a variable can be undone, so the path still ends at + // the first state. The composition theorem is not about it. + let with_variable = [Claim::Reversal, Claim::ReversalNotStated]; + assert_eq!(replay_path_end(&with_variable), " → fs₀"); + let note = replay_composition_note(&with_variable); + assert!(note.contains("1 of the 2 here"), "{note}"); + + let unstated = [Claim::IrreversibleNotStated]; + assert!(replay_path_end(&unstated).contains("not fs₀")); + } + + /// The closing line of a replay says what the theorems it showed claim. + /// It claimed a reversal theorem for every operation, an obliteration + /// included. + #[test] + fn test_replay_summary_says_what_each_theorem_claims() { + use crate::proof_refs::Claim; + + let all_reversible = replay_summary(&[Claim::Reversal, Claim::Reversal]); + assert!( + all_reversible.starts_with("Each has a reversal theorem"), + "{all_reversible}" + ); + + let mixed = replay_summary(&[ + Claim::Reversal, + Claim::ReversalOfAWrite, + Claim::Irreversible, + ]); + assert!(!mixed.contains("Each has"), "{mixed}"); + assert!( + mixed.contains("With a reversal theorem about the model: 1."), + "{mixed}" + ); + assert!( + mixed.contains("(none is stated for a redirection): 1."), + "{mixed}" + ); + assert!( + mixed.contains( + "Not reversible, with a theorem that the operation is not injective in the model: 1." + ), + "{mixed}" + ); + assert!(!mixed.contains("no theorem stated"), "{mixed}"); + assert!(mixed.ends_with("Where a theorem is shown, the link from the model to this code is tested, not proved."), "{mixed}"); + + // A change to a variable has no theorem, and the replay says so + // rather than count it among the reversal theorems. + let with_variables = replay_summary(&[ + Claim::Reversal, + Claim::ReversalNotStated, + Claim::ReversalNotStated, + Claim::IrreversibleNotStated, + ]); + assert!(!with_variables.contains("Each has"), "{with_variables}"); + assert!( + with_variables.contains("With a reversal theorem about the model: 1."), + "{with_variables}" + ); + assert!( + with_variables + .contains("Reversed from the shell's own record, with no theorem stated: 2."), + "{with_variables}" + ); + assert!( + with_variables.contains("Not reversible, with no theorem stated: 1."), + "{with_variables}" + ); + + // A replay of changes to variables alone claims no theorem at all. + let only_variables = replay_summary(&[Claim::ReversalNotStated]); + assert!( + only_variables.starts_with("With a reversal theorem about the model: 0."), + "{only_variables}" + ); + } + + /// The note beside the theorem of one step says what the theorem is + /// about, and says nothing where the theorem is a reversal theorem for + /// the operation itself. + #[test] + fn test_replay_note_for_each_claim() { + use crate::proof_refs::Claim; + + assert_eq!(replay_note(Claim::Reversal), ""); + assert!(replay_note(Claim::ReversalOfAWrite).contains("not for a redirection")); + assert!(replay_note(Claim::ReversalNotStated).contains("its own record")); + assert!(replay_note(Claim::Irreversible).contains("not injective in the model")); + assert!(replay_note(Claim::Irreversible).contains("cannot be reversed")); + assert!(replay_note(Claim::IrreversibleNotStated).contains("cannot be reversed")); + assert!(!replay_note(Claim::IrreversibleNotStated).contains("injective")); + } + #[test] fn test_parse_chmod_octal_755() { let result = parse_chmod_mode("755", 0o100644).unwrap(); diff --git a/impl/rust-cli/src/executable.rs b/impl/rust-cli/src/executable.rs index 80685c1..8841e34 100644 --- a/impl/rust-cli/src/executable.rs +++ b/impl/rust-cli/src/executable.rs @@ -1019,6 +1019,8 @@ impl ExecutableCommand for Command { ExecutionResult::Return { .. } => return Ok(cond_result), ExecutionResult::ExternalCommand { exit_code } => exit_code, }; + // What follows the condition sees its status as `$?`. + state.last_exit_code = cond_exit; if cond_exit == 0 { // Condition succeeded — execute then body @@ -1034,6 +1036,7 @@ impl ExecutableCommand for Command { ExecutionResult::Return { .. } => return Ok(elif_result), ExecutionResult::ExternalCommand { exit_code } => exit_code, }; + state.last_exit_code = elif_exit; if elif_exit == 0 { return execute_block(elif_body, state); @@ -1132,6 +1135,10 @@ impl ExecutableCommand for Command { ExecutionResult::Return { .. } => return Ok(left_result), ExecutionResult::ExternalCommand { exit_code } => exit_code, }; + // The right side sees the status of the left as `$?`: + // `false || echo $?` prints 1. It printed the status of + // whatever had run before the list. + state.last_exit_code = left_exit_code; // Short-circuit evaluation based on operator match operator { @@ -1673,6 +1680,10 @@ fn execute_function_call( Err(e) => { eprintln!("{}: {}", func_def.name, e); state.last_exit_code = 1; + // The function's own status too: a body whose last line + // could not be read returned the result of the line before + // it, so the call counted as a success. + last_result = ExecutionResult::ExternalCommand { exit_code: 1 }; } } } @@ -1762,6 +1773,7 @@ fn execute_loop( } ExecutionResult::ExternalCommand { exit_code } => exit_code, }; + state.last_exit_code = cond_exit; let should_continue = match kind { LoopKind::While => cond_exit == 0, LoopKind::Until => cond_exit != 0, diff --git a/impl/rust-cli/src/help.rs b/impl/rust-cli/src/help.rs index ed77855..2c3131a 100644 --- a/impl/rust-cli/src/help.rs +++ b/impl/rust-cli/src/help.rs @@ -319,7 +319,7 @@ fn display_command_list() -> Result<()> { "help ".bright_cyan() ); println!( - "The built-in file commands take one operand each (two for cp, mv and ln -s) and no options." + "The built-in file commands take one operand each; cp, mv and ln -s take two, and chmod and chown take a mode or an owner and then one file. None takes an option, apart from the -s of ln -s." ); Ok(()) diff --git a/impl/rust-cli/src/main.rs b/impl/rust-cli/src/main.rs index 7f8f99e..c459bb1 100644 --- a/impl/rust-cli/src/main.rs +++ b/impl/rust-cli/src/main.rs @@ -236,16 +236,7 @@ fn main() -> Result<()> { } } - // Run EXIT trap if registered - if let Some(exit_cmd) = state - .traps - .get(vsh::posix_builtins::TrapSignal::Exit) - .map(|s| s.to_string()) - { - if let Ok(cmd) = vsh::parser::parse_command(&exit_cmd) { - let _ = cmd.execute(&mut state); - } - } + run_exit_trap(&mut state); if interactive { std::process::exit(state.last_exit_code); @@ -254,6 +245,37 @@ fn main() -> Result<()> { Ok(()) } +/// Run the EXIT trap, if one is set. +/// +/// The text of a trap is a list of commands, as a script is: +/// `trap 'echo a; echo b' EXIT` runs both. It used to be read as one +/// command, so `echo` was given `a; echo b` as its words. A statement that +/// cannot be read is reported and the rest still run; `exit` in the trap +/// ends it. +fn run_exit_trap(state: &mut state::ShellState) { + let Some(body) = state + .traps + .get(vsh::posix_builtins::TrapSignal::Exit) + .map(|s| s.to_string()) + else { + return; + }; + for statement in vsh::parser::split_on_statement_separators(&body) { + let statement = statement.trim(); + if statement.is_empty() { + continue; + } + match vsh::parser::parse_command(statement) { + Ok(cmd) => { + if matches!(cmd.execute(state), Ok(ExecutionResult::Exit)) { + break; + } + } + Err(e) => eprintln!("vsh: trap: {e}"), + } + } +} + /// Execute script content (string of commands, one per line or semicolon-separated) /// /// We split the *entire* content on top-level statement separators (both `;` @@ -306,16 +328,7 @@ fn execute_script_content(content: &str, state: &mut state::ShellState) -> Resul } } - // Run EXIT trap if registered - if let Some(exit_cmd) = state - .traps - .get(vsh::posix_builtins::TrapSignal::Exit) - .map(|s| s.to_string()) - { - if let Ok(cmd) = vsh::parser::parse_command(&exit_cmd) { - let _ = cmd.execute(state); - } - } + run_exit_trap(state); std::process::exit(state.last_exit_code); } diff --git a/impl/rust-cli/src/parser.rs b/impl/rust-cli/src/parser.rs index 68e92c5..bda24c7 100644 --- a/impl/rust-cli/src/parser.rs +++ b/impl/rust-cli/src/parser.rs @@ -3372,14 +3372,16 @@ fn step_count(name: &str, args: &[String]) -> Result { [] => Ok(1), [count] => match count.parse::() { Ok(n) if n > 0 => Ok(n), - _ => Err(anyhow!( + _ => Err(Refusal(format!( "{name}: '{count}' is not a count of operations; nothing was done" - )), + )) + .into()), }, - _ => Err(anyhow!( + _ => Err(Refusal(format!( "{name}: takes one count, but {} arguments were given; nothing was done", args.len() - )), + )) + .into()), } } diff --git a/impl/rust-cli/src/proof_refs.rs b/impl/rust-cli/src/proof_refs.rs index 4667896..5cb660b 100644 --- a/impl/rust-cli/src/proof_refs.rs +++ b/impl/rust-cli/src/proof_refs.rs @@ -8,6 +8,29 @@ use crate::state::OperationType; use colored::Colorize; +/// What the theorem shown for an operation claims about it. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Claim { + /// The operation can be reversed, and the theorem is about it. + Reversal, + /// The operation can be reversed. The theorem shown is about a write of + /// a whole file: none is stated for a redirection, which is reported + /// under it. + ReversalOfAWrite, + /// The operation can be reversed: the shell does it from its own + /// record. No proof file states a theorem about it. + ReversalNotStated, + /// The operation cannot be reversed, and the theorem says that it is + /// not injective in the model. + Irreversible, + /// The operation cannot be reversed. No proof file states a theorem + /// about it. + IrreversibleNotStated, +} + +/// Shown in place of a theorem name where no proof file states one. +pub const NO_THEOREM: &str = "none stated"; + /// A reference to formal proofs across multiple verification systems. /// /// Maps each operation to its corresponding theorems in Coq, Lean 4, Agda, and Isabelle. @@ -50,6 +73,29 @@ impl ProofReference { /// /// # Returns /// The corresponding [`ProofReference`] with theorem locations + /// What the theorem that [`Self::for_operation`] gives claims about an + /// operation of type `op`. See [`Claim`]. + pub fn claim_for(op: OperationType) -> Claim { + let stated = Self::for_operation(op).is_stated(); + match (op.inverse().is_some(), stated) { + (true, true) => match op { + OperationType::FileTruncated | OperationType::FileAppended => { + Claim::ReversalOfAWrite + } + _ => Claim::Reversal, + }, + (true, false) => Claim::ReversalNotStated, + (false, true) => Claim::Irreversible, + (false, false) => Claim::IrreversibleNotStated, + } + } + + /// Whether a proof file states a theorem for this entry. An entry for + /// which none does has [`NO_THEOREM`] in place of a name. + pub fn is_stated(&self) -> bool { + self.theorem != NO_THEOREM + } + pub fn for_operation(op: OperationType) -> Self { match op { OperationType::Mkdir | OperationType::Rmdir => MKDIR_RMDIR_REVERSIBLE, @@ -62,9 +108,7 @@ impl ProofReference { OperationType::Symlink | OperationType::Unlink => SYMLINK_UNLINK_REVERSIBLE, OperationType::Chmod => CHMOD_REVERSIBLE, OperationType::Chown => CHOWN_REVERSIBLE, - OperationType::SetVariable | OperationType::UnsetVariable => { - VARIABLE_ASSIGNMENT_REVERSIBLE - } + OperationType::SetVariable | OperationType::UnsetVariable => VARIABLE_CHANGE_NO_THEOREM, OperationType::HardwareErase => HARDWARE_ERASE_IRREVERSIBLE, OperationType::Obliterate => OBLITERATE_IRREVERSIBLE, } @@ -72,8 +116,12 @@ impl ProofReference { /// Format as a short one-line reference for terminal display. /// - /// Returns: `theorem_name (coq_location)` with ANSI colors + /// Returns: `theorem_name (coq_location)` with ANSI colors, or + /// [`NO_THEOREM`] alone for an entry that has no theorem. pub fn format_short(&self) -> String { + if !self.is_stated() { + return self.theorem.bright_cyan().to_string(); + } format!( "{} ({})", self.theorem.bright_cyan(), @@ -213,46 +261,54 @@ pub const CHOWN_REVERSIBLE: ProofReference = ProofReference { description: "chown(old_uid/gid, chown(new_uid/gid, path, fs)) = fs when preconditions hold", }; -/// Proof reference for variable assignment reversibility. +/// Entry for a change to a shell variable. /// -/// Theorem: `set(name, old_value, set(name, new_value, env)) = env` -/// when previous value is captured. -/// This is a state-machine property: capturing pre-state enables reversal. -pub const VARIABLE_ASSIGNMENT_REVERSIBLE: ProofReference = ProofReference { - theorem: "variable_assignment_reversible", - coq_location: "proofs/coq/environment_model.v:pending", - lean_location: "proofs/lean4/EnvironmentModel.lean:pending", - agda_location: "proofs/agda/EnvironmentModel.agda:pending", +/// No proof file states a theorem about it. The entry used to name +/// `variable_assignment_reversible` and three files that do not exist. The +/// shell reverses the change from its own record of the earlier value. +pub const VARIABLE_CHANGE_NO_THEOREM: ProofReference = ProofReference { + theorem: NO_THEOREM, + coq_location: "none", + lean_location: "none", + agda_location: "none", isabelle_location: "none (no Isabelle theory was written before the archive)", - description: "set(name, old, set(name, new, env)) = env (pre-state capture)", + description: "no proof file states a theorem about a change to a variable; \ + the shell restores the earlier value from its own record", }; -/// Proof reference for hardware secure erase irreversibility. +/// Proof reference for a hardware erase. /// -/// Theorem: Hardware erase operations have NO inverse - data destruction is permanent. -/// Complies with NIST SP 800-88 Rev. 1 Purge methods. +/// The theorem is stated in the Idris2 layer only, where a hardware erase +/// is a function that gives the empty filesystem from every start. It says +/// that any two starts give the same result, so the erase is not injective +/// and nothing can recover the start. Coq, Lean and Agda state nothing +/// about a hardware erase. The theorem is about that model: it does not +/// prove that a device was erased, nor compliance with NIST SP 800-88. pub const HARDWARE_ERASE_IRREVERSIBLE: ProofReference = ProofReference { - theorem: "hardware_erase_irreversible", - coq_location: "proofs/coq/rmo_operations.v:L45-L68", - lean_location: "proofs/lean4/RMOOperations.lean:L52-L73", - agda_location: "proofs/agda/RMOOperations.agda:L48-L70", + theorem: "hardwareEraseIrreversible", + coq_location: "none (Idris2 only: proofs/idris2/src/Filesystem/RMO.idr:L238)", + lean_location: "none (Idris2 only: proofs/idris2/src/Filesystem/RMO.idr:L238)", + agda_location: "none (Idris2 only: proofs/idris2/src/Filesystem/RMO.idr:L238)", isabelle_location: "none (the archived RMO_Operations.thy has no hardware-erase theorem)", - description: "∀ fs device. ¬∃ op. apply(op, hardware_erase(device, fs)) = fs", + description: "∀ fs1 fs2. hardware_erase(fs1) = hardware_erase(fs2): the erase is not injective", }; -/// Proof reference for abstract obliterate irreversibility. +/// Proof reference for an obliteration. /// -/// The theorem is model-level: an obliterate transition has no inverse in the -/// abstract filesystem. It does not prove physical-media erasure, correspondence -/// with the Rust I/O sequence, or compliance with a legal regime. +/// The theorem is about the model: two stores that differ only in the data +/// of the blocks of the file give the same result once the file is +/// obliterated with at least one overwrite pattern. The operation is +/// therefore not injective, and nothing can recover the start. It does not +/// prove physical-media erasure, correspondence with the Rust I/O sequence, +/// or compliance with a legal regime. Agda states no such theorem. pub const OBLITERATE_IRREVERSIBLE: ProofReference = ProofReference { - theorem: "obliterate_irreversible", - coq_location: "proofs/coq/rmo_operations.v:L70-L95", - lean_location: "proofs/lean4/RMOOperations.lean:L75-L98", - agda_location: "proofs/agda/RMOOperations.agda:L72-L96", - isabelle_location: - "archive/proofs/isabelle/RMO_Operations.thy:L158-L180 (obliterate_not_injective)", - description: "∀ fs file. ¬∃ op. apply(op, obliterate(file, fs)) = fs", + theorem: "obliterate_not_injective", + coq_location: "proofs/coq/rmo_operations.v:L530", + lean_location: "proofs/lean4/RMOOperations.lean:L301", + agda_location: "none (RMOOperations.agda states no theorem of non-injectivity)", + isabelle_location: "archive/proofs/isabelle/RMO_Operations.thy:L158", + description: "stores that differ only in the block data of the file give the same \ + result: obliterate is not injective", }; /// Get all available proof references as a vector. @@ -350,3 +406,147 @@ pub fn print_verification_summary() { println!(" - Refinement proof from abstract to concrete"); println!(" - Verified compilation chain"); } + +#[cfg(test)] +mod tests { + use super::*; + + /// An operation is said to be irreversible exactly when it has no + /// inverse, so the two tables cannot drift apart. + #[test] + fn test_a_claim_of_irreversibility_matches_the_inverse_table() { + let all = [ + OperationType::Mkdir, + OperationType::Rmdir, + OperationType::CreateFile, + OperationType::DeleteFile, + OperationType::WriteFile, + OperationType::FileTruncated, + OperationType::FileAppended, + OperationType::CopyFile, + OperationType::Move, + OperationType::Symlink, + OperationType::Unlink, + OperationType::SetVariable, + OperationType::UnsetVariable, + OperationType::Chmod, + OperationType::Chown, + OperationType::HardwareErase, + OperationType::Obliterate, + ]; + for op in all { + let claim = ProofReference::claim_for(op); + assert_eq!( + matches!(claim, Claim::Irreversible | Claim::IrreversibleNotStated), + op.inverse().is_none(), + "{op:?}" + ); + assert_eq!( + matches!( + claim, + Claim::ReversalNotStated | Claim::IrreversibleNotStated + ), + !ProofReference::for_operation(op).is_stated(), + "{op:?}" + ); + } + } + + /// What a replay may claim for each kind of operation. A change to a + /// variable was shown under a reversal theorem that no proof file + /// states. + #[test] + fn test_the_claim_for_each_kind_of_operation() { + let claim = ProofReference::claim_for; + assert_eq!(claim(OperationType::Mkdir), Claim::Reversal); + assert_eq!(claim(OperationType::Move), Claim::Reversal); + assert_eq!(claim(OperationType::FileTruncated), Claim::ReversalOfAWrite); + assert_eq!(claim(OperationType::FileAppended), Claim::ReversalOfAWrite); + assert_eq!(claim(OperationType::SetVariable), Claim::ReversalNotStated); + assert_eq!( + claim(OperationType::UnsetVariable), + Claim::ReversalNotStated + ); + assert_eq!(claim(OperationType::Obliterate), Claim::Irreversible); + assert_eq!(claim(OperationType::HardwareErase), Claim::Irreversible); + } + + /// An entry with no theorem shows that, and no location beside it. + #[test] + fn test_an_entry_with_no_theorem_shows_no_location() { + let short = VARIABLE_CHANGE_NO_THEOREM.format_short(); + assert!(short.contains(NO_THEOREM), "{short}"); + assert!(!short.contains('('), "{short}"); + + let stated = OBLITERATE_IRREVERSIBLE.format_short(); + assert!(stated.contains("obliterate_not_injective"), "{stated}"); + assert!( + stated.contains("proofs/coq/rmo_operations.v:L530"), + "{stated}" + ); + } + + /// The path of the proof file that a location names, if it names one. + fn named_file(location: &str) -> Option<&str> { + let rest = &location[location.find("proofs/")?..]; + let end = rest.find([':', ')', ' ']).unwrap_or(rest.len()); + Some(&rest[..end]) + } + + /// Whether `text` declares something called `theorem`: after `Theorem`, + /// `Lemma`, `theorem` or `lemma` (Coq and Lean), or at the start of a + /// line before a colon (Idris2, and Agda with hyphens for underscores). + fn declares(text: &str, theorem: &str) -> bool { + let hyphenated = theorem.replace('_', "-"); + text.lines().any(|line| { + let mut words = line.split_whitespace(); + match (words.next(), words.next()) { + (Some("Theorem" | "Lemma" | "theorem" | "lemma"), Some(name)) => { + name.trim_end_matches(':') == theorem + } + (Some(name), Some(":")) => name == theorem || name == hyphenated, + _ => false, + } + }) + } + + /// Every theorem name the shell shows is declared in a proof file that + /// its entry names, and an entry with no theorem names no file. Three + /// entries failed this: the names shown for a change to a variable, for + /// a hardware erase and for an obliteration were declared in no file. + /// + /// The check is of names against files. It does not check the line + /// numbers in the entries, nor that each of Coq, Lean and Agda uses the + /// name shown. + #[test] + fn test_a_shown_theorem_is_declared_in_a_file_its_entry_names() { + let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."); + let mut entries = all_proofs(); + entries.extend([ + VARIABLE_CHANGE_NO_THEOREM, + HARDWARE_ERASE_IRREVERSIBLE, + OBLITERATE_IRREVERSIBLE, + ]); + assert_eq!(entries.len(), 12); + for entry in entries { + let files: Vec<&str> = [entry.coq_location, entry.lean_location, entry.agda_location] + .into_iter() + .filter_map(named_file) + .collect(); + let declared = files.iter().any(|file| { + let text = std::fs::read_to_string(root.join(file)) + .unwrap_or_else(|e| panic!("{file} cannot be read: {e}")); + declares(&text, entry.theorem) + }); + assert_eq!( + entry.is_stated(), + declared, + "{} in {files:?}", + entry.theorem + ); + if !entry.is_stated() { + assert!(files.is_empty(), "{files:?}"); + } + } + } +} diff --git a/impl/rust-cli/tests/builtin_strictness_tests.rs b/impl/rust-cli/tests/builtin_strictness_tests.rs index 1baea9d..0defa1c 100644 --- a/impl/rust-cli/tests/builtin_strictness_tests.rs +++ b/impl/rust-cli/tests/builtin_strictness_tests.rs @@ -425,3 +425,60 @@ fn a_malformed_line_is_still_a_parse_error() { Ok(Parsed::Refused { .. }) )); } + +/// What follows a command sees its status as `$?`: the right side of `||` +/// and `&&`, and the branches of an `if`. They used to see the status of +/// whatever had run before the whole list, so `false || echo $?` printed 0. +#[test] +fn what_follows_a_command_sees_its_status() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh( + dir.path(), + "false || echo \"left $?\"; true && echo \"left $?\"; \ + if false; then echo yes; else echo \"condition $?\"; fi; \ + mkdir -p x || echo \"refused $?\"", + ); + + assert_eq!( + run.stdout, "left 1\nleft 0\ncondition 1\nrefused 1\n", + "stderr: {}", + run.stderr + ); + + // The condition of an `elif` and of a loop is published in the same way. + let run = vsh( + dir.path(), + "if false; then echo a; elif true; then echo \"elif $?\"; fi; \ + until false; do echo \"loop $?\"; exit 0; done", + ); + + assert_eq!(run.stdout, "elif 0\nloop 1\n", "stderr: {}", run.stderr); +} + +/// A count that `undo` or `redo` cannot read is a refusal like the others: +/// the command fails and the line goes on. It used to end the line. +#[test] +fn a_count_that_cannot_be_read_fails_only_its_own_command() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh( + dir.path(), + "undo many || echo first; redo 1 2 || echo second", + ); + + assert_eq!(run.stdout, "first\nsecond\n", "stderr: {}", run.stderr); + assert_eq!(run.code, 0); +} + +/// The text of an EXIT trap is a list of commands, and each of them runs. +/// It used to be read as one command: `echo` was given `first; echo second`. +#[test] +fn an_exit_trap_runs_every_statement_of_its_text() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh(dir.path(), "trap 'echo first; echo second' EXIT; exit 3"); + + assert_eq!(run.stdout, "first\nsecond\n", "stderr: {}", run.stderr); + assert_eq!(run.code, 3); +} diff --git a/impl/rust-cli/tests/conformance/README.adoc b/impl/rust-cli/tests/conformance/README.adoc index d55c637..40563c6 100644 --- a/impl/rust-cli/tests/conformance/README.adoc +++ b/impl/rust-cli/tests/conformance/README.adoc @@ -17,7 +17,8 @@ end can do equals what this battery shows it does. is a comment object. | `known_failing.txt` -| The ids of the cases that do not match today. A ratchet: it may only shrink. +| The ids of the cases that do not match today. A ratchet: a case that has + matched never goes onto it (see "The ratchet" and "Adding a case"). | `../conformance_battery.rs` | The test. It starts the built `vsh` on each case in a fresh directory and @@ -85,9 +86,14 @@ operand. A case must be able to fail: check that its set-up runs. . Append a line to `battery.jsonl` with a new id of the form `group-NN`. . Take `out` and `rc` from a reference shell run in an empty directory. -. Run the test. If the case fails, the shell has a gap: add the id to - `known_failing.txt` and raise `KNOWN_FAILING_COUNT` in the same change, and - say so. +. Run the test. If the case fails, the shell has a gap. Fix the shell before + the case is merged. If the fix does not belong to the work in hand, keep the + case out of `battery.jsonl` for now, and record its command line and the + reference result with the work that will close the gap. + +A new case does not go onto `known_failing.txt`, and `KNOWN_FAILING_COUNT` is +not raised for one. The list is the set of failures measured when the battery +was added, and it only shrinks. == Limits diff --git a/impl/rust-cli/tests/conformance_battery.rs b/impl/rust-cli/tests/conformance_battery.rs index 04482f4..c71f1f4 100644 --- a/impl/rust-cli/tests/conformance_battery.rs +++ b/impl/rust-cli/tests/conformance_battery.rs @@ -175,6 +175,9 @@ fn run(case: &Case) -> Outcome { .arg("-c") .arg(&case.cmd) .current_dir(dir.path()) + // A home of the test's own, so that a case that reads `$HOME` or + // `~` gives the same answer on every machine. + .env("HOME", dir.path()) .env("NO_COLOR", "1") .env_remove("RUST_BACKTRACE") .stdin(Stdio::null()) diff --git a/impl/rust-cli/tests/function_control_flow_tests.rs b/impl/rust-cli/tests/function_control_flow_tests.rs index b51c6c0..c5c043a 100644 --- a/impl/rust-cli/tests/function_control_flow_tests.rs +++ b/impl/rust-cli/tests/function_control_flow_tests.rs @@ -183,3 +183,36 @@ fn function_body_with_brace_in_quoted_string_parses() -> Result<()> { assert!(state.resolve_path("ran").exists()); Ok(()) } + +// ------------------------------------------------------------------------- +// A body line that cannot be read +// ------------------------------------------------------------------------- + +/// A function whose last body line cannot be read fails. The call used to +/// return the result of the line before it, so it counted as a success. +#[test] +fn a_body_line_that_cannot_be_read_fails_the_function() -> Result<()> { + use vsh::executable::ExecutionResult; + use vsh::functions::{FunctionDef, SourceLocation}; + + let temp = tempdir()?; + let mut state = ShellState::new(temp.path().to_str().unwrap())?; + let raw_body = "true\necho \"unclosed".to_string(); + state.functions.define(FunctionDef { + name: "broken".to_string(), + body: vec![raw_body.clone()], + raw_body, + source_location: SourceLocation { + source: "test".to_string(), + line: 1, + }, + }); + + let result = parse_command("broken")?.execute(&mut state)?; + + assert!( + matches!(result, ExecutionResult::ExternalCommand { exit_code: 1 }), + "the call returned {result:?}" + ); + Ok(()) +} From 1139903bfcb50818791a99058b0dee871b35541e Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 18 Nov 2026 00:39:13 +0000 Subject: [PATCH 13/14] fix(shell): an EXIT trap reports a statement that fails MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A statement of an EXIT trap that failed with an error was dropped without a word. It is now reported as `vsh: trap: …`, as a statement that cannot be read already was, and the statements after it still run. Checked: 880 tests pass and 15 are ignored, with and without OLDPWD and PWD; clippy and fmt are clean. One mutation that drops the error again fails the new test. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_01J4QvDjNR1ZoYKEPxSKE2nq --- impl/rust-cli/src/main.rs | 14 ++++++------- .../tests/builtin_strictness_tests.rs | 20 +++++++++++++++++++ 2 files changed, 26 insertions(+), 8 deletions(-) diff --git a/impl/rust-cli/src/main.rs b/impl/rust-cli/src/main.rs index c459bb1..6d296da 100644 --- a/impl/rust-cli/src/main.rs +++ b/impl/rust-cli/src/main.rs @@ -250,8 +250,8 @@ fn main() -> Result<()> { /// The text of a trap is a list of commands, as a script is: /// `trap 'echo a; echo b' EXIT` runs both. It used to be read as one /// command, so `echo` was given `a; echo b` as its words. A statement that -/// cannot be read is reported and the rest still run; `exit` in the trap -/// ends it. +/// cannot be read, or that fails with an error, is reported and the rest +/// still run; `exit` in the trap ends it. fn run_exit_trap(state: &mut state::ShellState) { let Some(body) = state .traps @@ -265,12 +265,10 @@ fn run_exit_trap(state: &mut state::ShellState) { if statement.is_empty() { continue; } - match vsh::parser::parse_command(statement) { - Ok(cmd) => { - if matches!(cmd.execute(state), Ok(ExecutionResult::Exit)) { - break; - } - } + let ran = vsh::parser::parse_command(statement).and_then(|cmd| cmd.execute(state)); + match ran { + Ok(ExecutionResult::Exit) => break, + Ok(_) => {} Err(e) => eprintln!("vsh: trap: {e}"), } } diff --git a/impl/rust-cli/tests/builtin_strictness_tests.rs b/impl/rust-cli/tests/builtin_strictness_tests.rs index 0defa1c..eae154d 100644 --- a/impl/rust-cli/tests/builtin_strictness_tests.rs +++ b/impl/rust-cli/tests/builtin_strictness_tests.rs @@ -482,3 +482,23 @@ fn an_exit_trap_runs_every_statement_of_its_text() { assert_eq!(run.stdout, "first\nsecond\n", "stderr: {}", run.stderr); assert_eq!(run.code, 3); } + +/// A statement of an EXIT trap that fails is reported, and the statements +/// after it still run. Its error was dropped without a word. +#[test] +fn an_exit_trap_reports_a_statement_that_fails() { + let dir = tempfile::tempdir().expect("temporary directory"); + + let run = vsh( + dir.path(), + "mkdir d\ntrap 'mkdir d; echo second' EXIT\necho body", + ); + + assert!( + run.stdout.ends_with("body\nsecond\n"), + "stdout: {}", + run.stdout + ); + assert!(run.stderr.contains("vsh: trap: "), "stderr: {}", run.stderr); + assert!(run.stderr.contains("exists"), "stderr: {}", run.stderr); +} From ce3af2f15c6c9a68029efce7e221cbdd56782591 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Wed, 18 Nov 2026 01:44:20 +0000 Subject: [PATCH 14/14] fix(shell): a command that fails fails only itself; set -e works (#243) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Stacked on #233. The base is #233's branch, so the diff also shows the commit of #240 (`0a03ecc`), which is merged in here. Once #233 and #240 are merged, retarget this to `main`; the only new commit is `86ab3fd`. A built-in that failed was an error that unwound through every command around it, to the script or the prompt: - `mkdir x || echo no` never ran `echo no`. - A loop stopped at its first failure. - A function stopped, and its caller stopped with it. - A `-c` script or a script file ended at the first built-in that failed. That last behaviour stood in for `set -e`, which was not implemented. `set -e` only stored a variable, so `set -e; false; echo after` printed `after`. Now every command is a boundary. A command that fails prints `vsh: ` and the reason, and gives status 1. Whatever ran it goes on. `set -e` is honoured. A simple command that fails ends the shell with its status, and the EXIT trap still runs. As in bash, `set -e` does not apply in these places: - the condition of `if`, `elif`, `while` and `until`; - a command of an `&&` or `||` list before its last command; - a function called from either of those. `! cmd` is still not implemented. The owner chose this route after a measured prototype, on the condition that it lands together with a real `set -e`. Without it, scripts would lose the accidental stop at the first failure and become less safe. ## Changes - `impl/rust-cli/src/executable.rs`: - `ExecutableCommand::execute` is the boundary. It turns a failure into status 1 and applies `set -e`. The old body is now `Command::run`. - `as_condition` opens and closes the contexts in which `set -e` does not apply. - Three statuses change to what bash gives: - `return` outside a function: 2. - `eval` of a line that cannot be read: 2. - an `if` with no branch taken: 0, as POSIX requires. With `set -e`, the old 1 would end the shell. - `impl/rust-cli/src/state.rs`: two new fields, `errexit` and `errexit_suppressed`. - `impl/rust-cli/tests/command_boundary_tests.rs` (new): 28 lines run on the built shell, in four groups. Each expected result was recorded from bash 5.2.21. - Changed tests, each because the behaviour changed: - `function_and_script_tests.rs`: five checks of a status instead of an error from `execute`. - `function_frame_tests.rs`: the failure message reads `vsh: …`, not `Error: …`. - `builtin_strictness_tests.rs`: an EXIT trap statement that runs and fails is reported as `vsh: …`. - The battery list: `builtin-05` and `rm-02` now pass and come off `known_failing.txt`. The count in the test goes from 61 to 59. - `CLAUDE.md`, `README.adoc`, `ROADMAP.adoc`: - the battery count is now 69; - `set -e` is off the "does not work" list; - the line saying a failed built-in ends a `-c` script is gone. ## Measured, with bash 5.2.21 as reference | | before (#233 + #240) | this branch | |---|---|---| | 56 failure and `set -e` lines | 13 match bash | 53 match bash | | conformance battery, 128 lines | 67 match | 69 match; no case gets worse | The three lines that still differ are older faults this does not touch: - a redirection from a missing file gives 127; - `shift` is not a built-in; - the built-in `ls` gives 1 for a missing file where `ls` gives 2. dash agrees with bash on every case except the status cases noted in the test file: `return` and `local` outside a function, and `eval 'if'`, which end dash. ## RSR Quality Checklist ### Required - [x] Tests pass (`just test` or equivalent): `cargo test --no-fail-fast` in `impl/rust-cli/`: 889 passed, 0 failed, 15 ignored. That holds with `OLDPWD` and `PWD` set, with both unset, and with `--no-default-features`. `just test` itself was not run. - [x] Code is formatted (`just fmt` or equivalent): `cargo fmt --check`. - [x] Linter is clean (no new warnings or errors): `cargo clippy --all-targets -- -D warnings`, and the same with `--all-features`. - [x] No banned language patterns. - [x] No `unsafe` blocks without `// SAFETY:` comments (none added). - [x] No banned functions. - [x] SPDX license headers present on all new/modified source files. - [x] No secrets, credentials, or `.env` files included. ## Testing - **The four new tests fail without the change.** Against the code before this commit, the same test file fails all four. 21 of its 28 lines differ from bash there. - **The prompt tests**, which drive the shell through a pseudo-terminal, were run 400 times against each of the two prompts (the line editor and the plain prompt), with no failure. - **Not verified:** macOS; the stress suite; CI on this branch. ## Limits - `execute` keeps its `Result` type but no longer returns `Err` for a command that fails. A program using the library sees a status instead, and the message goes to standard error. - The 100,000-iteration safety limit on `while` and `until` now fails only the loop, with status 1, where before it ended the script. - No proof covers this. The evidence is tests only. --- _Generated by [Claude Code](https://claude.ai/code/session_01J4QvDjYffb3HcnwwDyrsF9)_ --------- Co-authored-by: Claude --- CLAUDE.md | 11 +- README.adoc | 2 +- ROADMAP.adoc | 2 +- impl/rust-cli/Cargo.toml | 5 + impl/rust-cli/src/executable.rs | 182 +++++++-- impl/rust-cli/src/state.rs | 11 + .../tests/builtin_strictness_tests.rs | 4 +- impl/rust-cli/tests/command_boundary_tests.rs | 170 +++++++++ .../tests/conformance/known_failing.txt | 2 - impl/rust-cli/tests/conformance_battery.rs | 2 +- .../tests/function_and_script_tests.rs | 64 +++- impl/rust-cli/tests/function_frame_tests.rs | 349 ++++++++++++++++++ 12 files changed, 742 insertions(+), 62 deletions(-) create mode 100644 impl/rust-cli/tests/command_boundary_tests.rs create mode 100644 impl/rust-cli/tests/function_frame_tests.rs diff --git a/CLAUDE.md b/CLAUDE.md index dd84010..f2feba6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,7 +24,7 @@ Opus audit session that fixed all broken tests, real bugs, and documentation. **Measured, not claimed (2026-11-17, gate G0).** `impl/rust-cli/tests/conformance_battery.rs` starts the built shell on 128 everyday command lines and compares the result with bash: -**67 match**. The cases that fail are listed in +**69 match**. The cases that fail are listed in `impl/rust-cli/tests/conformance/known_failing.txt`. That list is a ratchet: it may only shrink, and a case may not be added to it to make a build pass. @@ -40,7 +40,9 @@ Works through the built shell (passing battery cases, or checked by hand interac - `test`, `[ ... ]`, and `[[ ... ]]` on its own; `if/elif/else`, `while`, a single `for`, `case` - Functions, `local`, `return`; `$VAR`, `${VAR}`, `${VAR:-default}`, `${#VAR}`, `${VAR:o:n}` - `$((expr))`, and `$(cmd)` of another program, as the whole value of an assignment -- `echo`, `printf`, `eval`, `set --`, `alias`, `trap ... EXIT`, `exit N` +- `echo`, `printf`, `eval`, `set --`, `set -e`, `alias`, `trap ... EXIT`, `exit N` +- A command that fails fails only itself, with a status: the next command of a script, + a function or a loop runs, and `||` runs its right side - Reversible variable assignment (undo/redo) - Transaction grouping (`begin`/`commit`/`rollback`) - `explain`, `checkpoint`/`restore`, `diff`, `replay`, `history --proofs` @@ -53,14 +55,11 @@ In the source and unit-tested, but NOT working through the built shell (battery - `$((expr))` and `$(cmd)` inside an argument (`echo $((1+2))` prints the text); backquotes - A quoted `*`, `?`, `[`, `{` or `~` reaches other programs with a backslash added - `${VAR#pat}`, `${VAR%pat}`, `${VAR^^}` (give an empty string); `$$` -- Nested `for` loops (silently wrong), `break`, `continue`, `! cmd`, `set -e` +- Nested `for` loops (silently wrong), `break`, `continue`, `! cmd` - Subshells, brace groups in pipelines, arrays, `(( ))`, the C-style `for` - `export` and `VAR=value cmd` do not reach the program being run - Here documents passed with `-c`; redirections on built-in commands outside the session root (`ls nosuch 2>/dev/null`) -- A built-in that fails with an error (`mkdir` on a directory that exists, say) ends a `-c` - script, and the rest of an `&&` or `||` list with it. A refused built-in and `false` do - not: they fail with status 1 and the script goes on - Ctrl-Z does not suspend a running program Not exercised by the battery: process substitution, command correction, the pager. diff --git a/README.adoc b/README.adoc index 2269cfc..cb1fa3d 100644 --- a/README.adoc +++ b/README.adoc @@ -53,7 +53,7 @@ Valence Shell is a formally verified shell with proven reversibility guarantees. [NOTE] ==== -*Measured, not claimed.* `impl/rust-cli/tests/conformance_battery.rs` starts the built shell on 128 everyday command lines and compares the result with bash. On 2026-11-17, 67 match. The cases that fail are listed in `impl/rust-cli/tests/conformance/known_failing.txt`, which may only shrink. Below, `[x]` means it works through the built shell, `[~]` means in part, and the limit is stated. +*Measured, not claimed.* `impl/rust-cli/tests/conformance_battery.rs` starts the built shell on 128 everyday command lines and compares the result with bash. On 2026-11-18, 69 match. The cases that fail are listed in `impl/rust-cli/tests/conformance/known_failing.txt`, which may only shrink. Below, `[x]` means it works through the built shell, `[~]` means in part, and the limit is stated. ==== * [~] Built-in file commands with undo: `mkdir`, `rmdir`, `touch`, `rm`, `cp`, `mv`, `ln -s`, `chmod`, `chown`, in their plain forms only (one operand each; two for `cp`, `mv` and `ln -s`; a mode or an owner and then one file for `chmod` and `chown`; no options, the `-s` of `ln -s` apart). An option or an extra operand is refused and nothing is done diff --git a/ROADMAP.adoc b/ROADMAP.adoc index f7c3f98..ab03bbf 100644 --- a/ROADMAP.adoc +++ b/ROADMAP.adoc @@ -88,7 +88,7 @@ integration), #56/#57/#58 (the three admits surfaced by #55). [IMPORTANT] ==== -*Parked* by owner ruling OD-42 (2026-11-17; recorded in `docs/plans/VSH-ROUTE-PLAN-2026-11-04.adoc` on valence-shell#230). The engine is the product and the shell interpreter is one front end, which comes last. What the interpreter does today is measured by `impl/rust-cli/tests/conformance_battery.rs` (67 of 128 command lines match bash), not by the list below, parts of which are out of date. +*Parked* by owner ruling OD-42 (2026-11-17; recorded in `docs/plans/VSH-ROUTE-PLAN-2026-11-04.adoc` on valence-shell#230). The engine is the product and the shell interpreter is one front end, which comes last. What the interpreter does today is measured by `impl/rust-cli/tests/conformance_battery.rs` (69 of 128 command lines match bash), not by the list below, parts of which are out of date. ==== Most critical missing POSIX features, ranked by impact (per CLAUDE.md diff --git a/impl/rust-cli/Cargo.toml b/impl/rust-cli/Cargo.toml index d0a8569..e2a4525 100644 --- a/impl/rust-cli/Cargo.toml +++ b/impl/rust-cli/Cargo.toml @@ -84,6 +84,11 @@ which = "7.0" proptest = "1.10" tempfile = "3.26" +# Pseudo-terminal for the tests that run the built shell at its prompt +# (tests/function_frame_tests.rs). Same crate and version as above; the `term` +# feature is turned on for test builds only. +nix = { version = "0.31", default-features = false, features = ["fs", "term"] } + # Benchmarking criterion = "0.5" diff --git a/impl/rust-cli/src/executable.rs b/impl/rust-cli/src/executable.rs index 8841e34..68bc6b0 100644 --- a/impl/rust-cli/src/executable.rs +++ b/impl/rust-cli/src/executable.rs @@ -70,8 +70,10 @@ pub enum ExecutionResult { Return { exit_code: i32 }, } -impl ExecutableCommand for Command { - fn execute(&self, state: &mut ShellState) -> Result { +impl Command { + /// Run the command itself. A failure can come back as an error here; + /// [`ExecutableCommand::execute`] turns it into a failed status. + fn run(&self, state: &mut ShellState) -> Result { match self { // Filesystem operations (reversible) Command::Mkdir { path, redirects } => { @@ -949,16 +951,13 @@ impl ExecutableCommand for Command { let params: Vec = args[1..].to_vec(); state.set_positional_params(params); } else if args[0].starts_with('-') || args[0].starts_with('+') { - // Shell options (stub: just acknowledge) - // -e: exit on error, -x: trace, -u: error on unset, etc. + // Shell options. `-e` is honoured; `-x` and `-u` are only + // recorded. for arg in args { let enable = arg.starts_with('-'); for ch in arg[1..].chars() { match ch { - 'e' => state.set_variable( - "SHOPT_E".to_string(), - if enable { "1" } else { "0" }.to_string(), - ), + 'e' => state.errexit = enable, 'x' => state.set_variable( "SHOPT_X".to_string(), if enable { "1" } else { "0" }.to_string(), @@ -1000,8 +999,15 @@ impl ExecutableCommand for Command { .map(|a| crate::parser::expand_variables(a, state)) .collect(); let combined = expanded_args.join(" "); - let cmd = crate::parser::parse_command(&combined)?; - cmd.execute(state) + match crate::parser::parse_command(&combined) { + Ok(cmd) => cmd.execute(state), + // A line that cannot be read is status 2, as in bash + // and dash. + Err(e) => { + eprintln!("vsh: eval: {e}"); + Ok(ExecutionResult::ExternalCommand { exit_code: 2 }) + } + } } // Control structures @@ -1011,8 +1017,9 @@ impl ExecutableCommand for Command { elif_parts, else_body, } => { - // Execute condition and get exit code - let cond_result = condition.execute(state)?; + // Execute condition and get exit code. `set -e` does not + // apply to it. + let cond_result = as_condition(state, |s| condition.execute(s))?; let cond_exit = match cond_result { ExecutionResult::Success => 0, ExecutionResult::Exit => return Ok(ExecutionResult::Exit), @@ -1029,7 +1036,7 @@ impl ExecutableCommand for Command { // Try elif parts for (elif_cond, elif_body) in elif_parts { - let elif_result = elif_cond.execute(state)?; + let elif_result = as_condition(state, |s| elif_cond.execute(s))?; let elif_exit = match elif_result { ExecutionResult::Success => 0, ExecutionResult::Exit => return Ok(ExecutionResult::Exit), @@ -1048,8 +1055,9 @@ impl ExecutableCommand for Command { return execute_block(else_cmds, state); } - // No branch matched — return last condition's exit code - Ok(ExecutionResult::ExternalCommand { exit_code: 1 }) + // No branch ran: the status is 0 (POSIX 2.9.4.4). It was 1, + // which `set -e` would take for a failure. + Ok(ExecutionResult::ExternalCommand { exit_code: 0 }) } Command::WhileLoop { condition, body } => { @@ -1125,8 +1133,9 @@ impl ExecutableCommand for Command { } => { use crate::parser::LogicalOperator; - // Execute left command - let left_result = left.execute(state)?; + // Execute left command. `set -e` does not apply to it, only + // to the command that ends the list. + let left_result = as_condition(state, |s| left.execute(s))?; // Get exit code from left command let left_exit_code = match left_result { @@ -1214,7 +1223,9 @@ impl ExecutableCommand for Command { // Return from function Command::Return { code } => { if state.functions.call_depth() == 0 { - anyhow::bail!("return: can only return from a function"); + // Status 2, as bash gives. + eprintln!("vsh: return: can only return from a function"); + return Ok(ExecutionResult::ExternalCommand { exit_code: 2 }); } let exit_code = code.unwrap_or(state.last_exit_code); state.last_exit_code = exit_code; @@ -1296,6 +1307,58 @@ impl ExecutableCommand for Command { } } + /// The commands that hold other commands. Their status is that of a + /// command inside them, which `set -e` has already seen, or of a + /// condition, which `set -e` does not apply to. + fn is_compound(&self) -> bool { + matches!( + self, + Command::If { .. } + | Command::WhileLoop { .. } + | Command::UntilLoop { .. } + | Command::ForLoop { .. } + | Command::CaseStatement { .. } + | Command::LogicalOp { .. } + | Command::Source { .. } + | Command::Eval { .. } + ) + } +} + +impl ExecutableCommand for Command { + /// Every command is a boundary. A command that fails prints what went + /// wrong and fails with status 1; whatever ran it goes on, as in a POSIX + /// shell: the next line of a script or a function, the next turn of a + /// loop, the other side of `||`. It used to be an error that unwound + /// through every enclosing command, so `mkdir x || echo no` never ran + /// `echo no`, and a script ended at the first built-in that failed. + /// + /// With `set -e`, a simple command that fails ends the shell, outside + /// the contexts listed at [`ShellState::errexit_suppressed`]. + /// + /// This never returns `Err`. The type stays a `Result` for the callers. + fn execute(&self, state: &mut ShellState) -> Result { + let result = match self.run(state) { + Ok(result) => result, + Err(e) => { + eprintln!("vsh: {e}"); + state.last_exit_code = 1; + ExecutionResult::ExternalCommand { exit_code: 1 } + } + }; + if let ExecutionResult::ExternalCommand { exit_code } = result { + if exit_code != 0 + && state.errexit + && state.errexit_suppressed == 0 + && !self.is_compound() + { + state.last_exit_code = exit_code; + return Ok(ExecutionResult::Exit); + } + } + Ok(result) + } + fn is_reversible(&self) -> bool { matches!( self, @@ -1608,11 +1671,12 @@ fn expand_operand(name: &str, raw: &str, state: &ShellState) -> Option { /// 1. Save current positional parameters /// 2. Set new positional parameters from function arguments /// 3. Push a function frame for local variable scoping -/// 4. Parse the raw function body, splitting on semicolons/newlines with -/// control-structure awareness (so `if/fi`, `for/done`, `while/done`, -/// and `case/esac` work inside a function body) -/// 5. Execute each segment; stop early on `return` / `exit` -/// 6. Restore positional parameters and local variables +/// 4. Run the body ([`run_function_body`]) +/// 5. Pop the frame: restore positional parameters and local variables +/// +/// Step 5 runs however the body ended: at its last command, at `return` or +/// `exit`, or at a command that returned an error. The error still goes to +/// the caller; the frame does not go with it. fn execute_function_call( func_def: &crate::functions::FunctionDef, args: &[String], @@ -1627,6 +1691,47 @@ fn execute_function_call( // Push a function frame state.functions.push_frame(saved_params.clone()); + // Run the body. There is no `?` on this call, and there must not be one: + // the frame is popped before the result goes back, whichever it is. An + // error that left with the frame still pushed would leave the caller + // with the function's local variables and positional parameters, and + // with a call depth that lets `local` and `return` run outside any + // function. + let result = run_function_body(func_def, state); + + // Pop function frame and restore local variables + if let Some(frame) = state.functions.pop_frame() { + // Restore positional parameters + state.set_positional_params(frame.saved_params); + + // Restore local variables to their previous values + for (var_name, prev_value) in &frame.saved_vars { + match prev_value { + Some(val) => state.set_variable(var_name.clone(), val.clone()), + None => { + state.unset_variable(var_name); + } + } + } + } + + result +} + +/// Run the body of a shell function, inside the frame that +/// [`execute_function_call`] pushed and will pop. +/// +/// 1. Parse the raw function body, splitting on semicolons/newlines with +/// control-structure awareness (so `if/fi`, `for/done`, `while/done`, +/// and `case/esac` work inside a function body) +/// 2. Execute each segment; stop early on `return` / `exit` +/// +/// This is a function of its own so that every way out of the body, the `?` +/// below included, comes back to one place in the caller. +fn run_function_body( + func_def: &crate::functions::FunctionDef, + state: &mut ShellState, +) -> Result { // Execute function body. We prefer the raw body string (which preserves // control-structure integrity) and fall back to the pre-split segments // for older in-memory definitions. @@ -1688,25 +1793,22 @@ fn execute_function_call( } } - // Pop function frame and restore local variables - if let Some(frame) = state.functions.pop_frame() { - // Restore positional parameters - state.set_positional_params(frame.saved_params); - - // Restore local variables to their previous values - for (var_name, prev_value) in &frame.saved_vars { - match prev_value { - Some(val) => state.set_variable(var_name.clone(), val.clone()), - None => { - state.unset_variable(var_name); - } - } - } - } - Ok(last_result) } +/// Run `f` where `set -e` does not apply: a condition, or a command of an +/// `&&` or `||` list that is not the last. The context closes on every way +/// out of `f`. +fn as_condition( + state: &mut ShellState, + f: impl FnOnce(&mut ShellState) -> Result, +) -> Result { + state.errexit_suppressed += 1; + let result = f(state); + state.errexit_suppressed -= 1; + result +} + /// Execute a block of commands in sequence, returning the result of the last command. /// /// Short-circuits on `Exit` (shell-wide) and `Return` (function-scope) so the @@ -1764,7 +1866,7 @@ fn execute_loop( } iterations += 1; - let cond_result = condition.execute(state)?; + let cond_result = as_condition(state, |s| condition.execute(s))?; let cond_exit = match cond_result { ExecutionResult::Success => 0, ExecutionResult::Exit => return Ok(ExecutionResult::Exit), diff --git a/impl/rust-cli/src/state.rs b/impl/rust-cli/src/state.rs index c48b1bf..291aecd 100644 --- a/impl/rust-cli/src/state.rs +++ b/impl/rust-cli/src/state.rs @@ -288,6 +288,15 @@ pub struct ShellState { /// loops. A script, or `-c`, leaves it unset. pub interactive: bool, + /// `set -e`: a simple command that fails ends the shell, with its status. + pub errexit: bool, + + /// How many contexts are open in which `set -e` does not apply: the + /// condition of `if`, `elif`, `while` and `until`, and every command of + /// an `&&` or `||` list but the last. A function called there is inside + /// the context too. + pub errexit_suppressed: usize, + /// Previous directory for cd - support pub previous_dir: Option, @@ -351,6 +360,8 @@ impl ShellState { state_file, last_exit_code: 0, interactive: false, + errexit: false, + errexit_suppressed: 0, previous_dir: None, variables: HashMap::new(), exported_vars: HashSet::new(), diff --git a/impl/rust-cli/tests/builtin_strictness_tests.rs b/impl/rust-cli/tests/builtin_strictness_tests.rs index eae154d..77f38a5 100644 --- a/impl/rust-cli/tests/builtin_strictness_tests.rs +++ b/impl/rust-cli/tests/builtin_strictness_tests.rs @@ -499,6 +499,8 @@ fn an_exit_trap_reports_a_statement_that_fails() { "stdout: {}", run.stdout ); - assert!(run.stderr.contains("vsh: trap: "), "stderr: {}", run.stderr); + // A statement that cannot be read says `vsh: trap:`. One that runs and + // fails is reported like any failed command, `vsh: ` and the reason. + assert!(run.stderr.contains("vsh: "), "stderr: {}", run.stderr); assert!(run.stderr.contains("exists"), "stderr: {}", run.stderr); } diff --git a/impl/rust-cli/tests/command_boundary_tests.rs b/impl/rust-cli/tests/command_boundary_tests.rs new file mode 100644 index 0000000..c6a5eba --- /dev/null +++ b/impl/rust-cli/tests/command_boundary_tests.rs @@ -0,0 +1,170 @@ +// SPDX-License-Identifier: MPL-2.0 +// Copyright (c) Jonathan D.A. Jewell +//! A command that fails fails only itself, and `set -e` ends the shell. +//! +//! A built-in that failed used to be an error that unwound through every +//! command around it: `mkdir x || echo no` never ran `echo no`, a loop +//! stopped at its first failure, and a script ended at the first built-in +//! that failed. That last one stood in for `set -e`, which was not +//! implemented: `set -e; false; echo after` printed `after`. +//! +//! Now each command is a boundary. A failure prints what went wrong and +//! gives a status, and whatever ran the command goes on. `set -e` ends the +//! shell at a simple command that fails, except in a condition and in an +//! `&&` or `||` list before its last command. +//! +//! Every expected output and status here is what GNU bash 5.2.21 gives for +//! the same line (`bash -c`, standard error not compared); dash 5.2 agrees +//! except where a case says otherwise. Each test runs the built shell. + +use std::process::{Command, Stdio}; + +/// Runs `script` with `vsh -c` in a fresh directory. Returns the standard +/// output, without the shell's own `[op:` receipt lines, and the status. +fn vsh(script: &str) -> (String, i32) { + let dir = tempfile::tempdir().expect("temporary directory"); + let output = Command::new(env!("CARGO_BIN_EXE_vsh")) + .arg("-r") + .arg(dir.path()) + .arg("-c") + .arg(script) + .current_dir(dir.path()) + .env("HOME", dir.path()) + .env("NO_COLOR", "1") + .env_remove("RUST_BACKTRACE") + .stdin(Stdio::null()) + .output() + .expect("start the built shell"); + let stdout = String::from_utf8_lossy(&output.stdout) + .split_inclusive('\n') + .filter(|line| !line.starts_with("[op:") && !line.starts_with("[txn:")) + .collect(); + (stdout, output.status.code().unwrap_or(-1)) +} + +/// Runs each case and reports every one that differs, not only the first. +fn check(cases: &[(&str, &str, i32)]) { + let wrong: Vec = cases + .iter() + .filter_map(|&(script, out, status)| { + let (got_out, got_status) = vsh(script); + (got_out != out || got_status != status).then(|| { + format!( + " {script}\n expected {out:?}, status {status}\n got \ + {got_out:?}, status {got_status}" + ) + }) + }) + .collect(); + assert!( + wrong.is_empty(), + "{} of {} cases differ from bash:\n{}", + wrong.len(), + cases.len(), + wrong.join("\n") + ); +} + +#[test] +fn a_command_that_fails_fails_only_itself() { + check(&[ + // The line from the review of #233. + ( + r#"f() { local x=inner; mkdir /no/such/parent/dir; }; x=outer; f; echo "$x""#, + "outer\n", + 0, + ), + ("mkdir no/such/dir; echo rc=$?", "rc=1\n", 0), + ("mkdir no/such/dir || echo fallback", "fallback\n", 0), + ("rm nosuch && echo no; echo rc=$?", "rc=1\n", 0), + ( + "for i in 1 2; do rm nosuch; echo body$i; done; echo end", + "body1\nbody2\nend\n", + 0, + ), + ( + "f() { rm nosuch; echo infunc; }; f; echo rc=$?", + "infunc\nrc=0\n", + 0, + ), + ( + "f() { rm nosuch; }; if f; then echo yes; else echo no; fi", + "no\n", + 0, + ), + ( + "case x in x) rm nosuch; echo arm;; esac; echo end", + "arm\nend\n", + 0, + ), + ( + "trap 'echo bye' EXIT; rm nosuch; echo after", + "after\nbye\n", + 0, + ), + ]); +} + +/// The status a failure gives. dash differs on the first three: it ends the +/// shell at `return` and `local` outside a function, and at the `eval`. +#[test] +fn a_failure_gives_the_status_bash_gives() { + check(&[ + ("return 0; echo rc=$?", "rc=2\n", 0), + ("eval 'if'; echo rc=$?", "rc=2\n", 0), + ("local y=1; echo rc=$?", "rc=1\n", 0), + // No branch ran: status 0. It was 1, which `set -e` would take for + // a failure. + ("if false; then :; fi; echo rc=$?", "rc=0\n", 0), + ]); +} + +#[test] +fn set_e_ends_the_shell_at_a_command_that_fails() { + check(&[ + ("set -e; false; echo after", "", 1), + ("set -e; rm nosuch; echo after", "", 1), + ("set -e; f() { false; echo infunc; }; f; echo after", "", 1), + ("set -e; true && false; echo after", "", 1), + ( + "set -e; for i in 1 2; do false; echo body; done; echo after", + "", + 1, + ), + // A function that returns a failure is a command that fails. + ("set -e; f() { return 3; }; f; echo after", "", 3), + // The EXIT trap still runs. + ( + "set -e; trap 'echo bye' EXIT; false; echo after", + "bye\n", + 1, + ), + ]); +} + +#[test] +fn set_e_does_not_apply_to_a_condition_or_inside_a_list() { + check(&[ + ("set -e; false && true; echo after", "after\n", 0), + ("set -e; false || true; echo after", "after\n", 0), + ( + "set -e; mkdir no/such/dir || true; echo after", + "after\n", + 0, + ), + ("set -e; if false; then :; fi; echo after", "after\n", 0), + ( + "set -e; if mkdir no/such/dir; then :; else echo no; fi; echo after", + "no\nafter\n", + 0, + ), + ("set -e; while false; do :; done; echo after", "after\n", 0), + // A function called as a condition is in the condition too. + ( + "set -e; f() { false; echo infunc; }; if f; then echo y; fi; echo after", + "infunc\ny\nafter\n", + 0, + ), + ("set -e; set +e; false; echo after", "after\n", 0), + ]); +} diff --git a/impl/rust-cli/tests/conformance/known_failing.txt b/impl/rust-cli/tests/conformance/known_failing.txt index 5c8a1e3..5176dff 100644 --- a/impl/rust-cli/tests/conformance/known_failing.txt +++ b/impl/rust-cli/tests/conformance/known_failing.txt @@ -50,7 +50,6 @@ sub-01 # mkdir zzdir; (cd zzdir; pwd | grep -c zzdir); pwd | grep -c zzdir sub-03 # { echo a; echo b; } | wc -l sub-04 # echo a & wait; echo done builtin-03 # read a b <<< "one two"; echo $b -builtin-05 # set -e; false; echo notreached builtin-06 # x=1; unset x; echo "[${x}]" builtin-07 # type echo | head -1 | grep -c echo builtin-12 # mkdir zzdir; pushd zzdir >/dev/null; pwd | grep -c zzdir; popd >/dev/null @@ -66,7 +65,6 @@ loop-10 # for i in a b; do for j in 1 2; do echo $i$j; done; done subst-05 # x=$(echo hi); echo $x fs-09 # touch a b c; ls | wc -l fs-10 # mkdir x y; ls | wc -l -rm-02 # rm nosuch; echo rc=$? rm-03 # rm -f nosuch; echo rc=$? rm-04 # mkdir d; touch d/x; rm -r d; ls | wc -l rm-05 # touch a; touch b; rm a b; ls | wc -l diff --git a/impl/rust-cli/tests/conformance_battery.rs b/impl/rust-cli/tests/conformance_battery.rs index c71f1f4..7aaf95f 100644 --- a/impl/rust-cli/tests/conformance_battery.rs +++ b/impl/rust-cli/tests/conformance_battery.rs @@ -54,7 +54,7 @@ const KNOWN_FAILING: &str = include_str!("conformance/known_failing.txt"); /// What this does not catch: one case taken off the list and another put /// on in the same change. Only a comparison with the base branch catches /// that. -const KNOWN_FAILING_COUNT: usize = 61; +const KNOWN_FAILING_COUNT: usize = 59; /// Longest a single case may run. A case that does not finish counts as failed. const TIME_LIMIT: Duration = Duration::from_secs(5); diff --git a/impl/rust-cli/tests/function_and_script_tests.rs b/impl/rust-cli/tests/function_and_script_tests.rs index 51801da..13cd0f9 100644 --- a/impl/rust-cli/tests/function_and_script_tests.rs +++ b/impl/rust-cli/tests/function_and_script_tests.rs @@ -158,6 +158,42 @@ fn test_function_local_variables() -> Result<()> { Ok(()) } +/// A command in the body fails. The function goes on to its end, and both +/// frames are popped. +/// +/// This test calls the library, so it runs on every platform. The tests that +/// run the built shell are in `function_frame_tests.rs` (Linux only). +#[test] +fn test_function_that_fails_pops_its_frame() -> Result<()> { + let temp = tempdir()?; + let mut state = ShellState::new(temp.path().to_str().unwrap())?; + + state.set_variable("X", "global"); + state.set_positional_params(vec!["outer_arg".to_string()]); + + // `inner` fails: the parent of the directory does not exist + let inner_cmd = parse_command("inner() { local X=inner_val; mkdir no/such/parent/dir; }")?; + inner_cmd.execute(&mut state)?; + let outer_cmd = parse_command("outer() { local X=outer_val; inner inner_arg; }")?; + outer_cmd.execute(&mut state)?; + + let invoke_cmd = parse_command("outer outer_arg_1 outer_arg_2")?; + let result = invoke_cmd.execute(&mut state)?; + + // The failed mkdir is the last command, so the call fails with status 1 + assert!( + matches!(result, ExecutionResult::ExternalCommand { exit_code: 1 }), + "{result:?}" + ); + + // Both frames are gone, and the caller has its own values back + assert_eq!(state.functions.call_depth(), 0); + assert_eq!(state.get_variable("X"), Some("global")); + assert_eq!(state.positional_params, vec!["outer_arg".to_string()]); + + Ok(()) +} + #[test] fn test_function_return_value() -> Result<()> { let temp = tempdir()?; @@ -187,11 +223,13 @@ fn test_return_outside_function_fails() -> Result<()> { let mut state = ShellState::new(temp.path().to_str().unwrap())?; let cmd = parse_command("return 0")?; - let result = cmd.execute(&mut state); + let result = cmd.execute(&mut state)?; - assert!(result.is_err()); - let err = result.unwrap_err(); - assert!(err.to_string().contains("can only return from a function")); + // A failed command with status 2, as in bash; not a return + assert!( + matches!(result, ExecutionResult::ExternalCommand { exit_code: 2 }), + "{result:?}" + ); Ok(()) } @@ -202,11 +240,14 @@ fn test_local_outside_function_fails() -> Result<()> { let mut state = ShellState::new(temp.path().to_str().unwrap())?; let cmd = parse_command("local X=5")?; - let result = cmd.execute(&mut state); + let result = cmd.execute(&mut state)?; - assert!(result.is_err()); - let err = result.unwrap_err(); - assert!(err.to_string().contains("can only be used in a function")); + // A failed command with status 1, as in bash, and nothing was set + assert!( + matches!(result, ExecutionResult::ExternalCommand { exit_code: 1 }), + "{result:?}" + ); + assert_eq!(state.get_variable("X"), None); Ok(()) } @@ -352,9 +393,12 @@ fn test_source_nonexistent_file_fails() -> Result<()> { let mut state = ShellState::new(temp.path().to_str().unwrap())?; let cmd = parse_command("source /nonexistent/file.sh")?; - let result = cmd.execute(&mut state); + let result = cmd.execute(&mut state)?; - assert!(result.is_err()); + assert!( + matches!(result, ExecutionResult::ExternalCommand { exit_code: 1 }), + "{result:?}" + ); Ok(()) } diff --git a/impl/rust-cli/tests/function_frame_tests.rs b/impl/rust-cli/tests/function_frame_tests.rs new file mode 100644 index 0000000..a56cccf --- /dev/null +++ b/impl/rust-cli/tests/function_frame_tests.rs @@ -0,0 +1,349 @@ +// SPDX-License-Identifier: MPL-2.0 +// Copyright (c) Jonathan D.A. Jewell +//! A function gives its caller's variables and positional parameters back, +//! also when a command in it fails. +//! +//! A call pushes a frame: the caller's positional parameters, and the old +//! value of each variable the body declares `local`. The frame was popped +//! when the body ran to its end or reached `return`. A command that returned +//! an error left through `?`, past the pop. The caller then kept the +//! function's `local` values, had the function's arguments as `$1`, `$2`, +//! ..., and the shell went on as if it were still inside the function. +//! +//! These tests run the built shell, and they run it as a person does: at the +//! prompt, through a pseudo-terminal. That is where the fault shows. The +//! prompt reports an error and reads the next line with the same state. With +//! `-c` or a script file the first error ends the whole run (another fault, +//! not this one), so no command is left to show what the caller kept. +//! +//! The `after:` lines expected here are what GNU bash 5.2.21 prints for the +//! same lines. +//! +//! Built on Linux only: the pseudo-terminal handling here has been run +//! nowhere else. + +#![cfg(target_os = "linux")] + +use std::fs::{self, File}; +use std::io::{Read, Write}; +use std::path::Path; +use std::process::{Command, Stdio}; +use std::sync::mpsc; +use std::time::{Duration, Instant}; + +use nix::fcntl::{fcntl, FcntlArg, FdFlag}; +use nix::pty::{openpty, Winsize}; + +/// The line editor asks the terminal where the cursor is before it draws the +/// prompt, and gives up after two seconds without an answer. +const CURSOR_QUERY: &[u8] = b"\x1b[6n"; + +/// What a terminal answers: row 1, column 1. +const CURSOR_ANSWER: &[u8] = b"\x1b[1;1R"; + +/// Every prompt of the shell ends like this, in both of its builds. +const PROMPT_END: &str = "> "; + +/// The longest a session may take. One takes well under a second. +const LIMIT: Duration = Duration::from_secs(30); + +/// Runs the built shell at its prompt in `dir`, types `lines` and then +/// `exit`, and returns everything the shell sent to the terminal. +/// +/// The shell has two prompts. The default build reads keys through a line +/// editor; a build with `--no-default-features` prints a prompt and reads a +/// line. Both are driven here, whichever was built. +fn at_the_prompt(dir: &Path, lines: &[&str]) -> String { + let size = Winsize { + ws_row: 24, + ws_col: 80, + ws_xpixel: 0, + ws_ypixel: 0, + }; + let pty = openpty(Some(&size), None).expect("open a pseudo-terminal"); + // Keep both ends out of the shells that other tests start meanwhile. + for end in [&pty.master, &pty.slave] { + fcntl(end, FcntlArg::F_SETFD(FdFlag::FD_CLOEXEC)).expect("set close-on-exec"); + } + let mut screen = File::from(pty.master); + let mut keyboard = screen.try_clone().expect("a second handle on the terminal"); + + let mut shell = { + let terminal = pty.slave; + let mut command = Command::new(env!("CARGO_BIN_EXE_vsh")); + command + .arg("-r") + .arg(dir) + .current_dir(dir) + // The line editor keeps its history file in the home directory. + .env("HOME", dir) + .env("NO_COLOR", "1") + .env("TERM", "xterm") + .env_remove("RUST_BACKTRACE") + .stdin(Stdio::from( + terminal.try_clone().expect("terminal for input"), + )) + .stdout(Stdio::from( + terminal.try_clone().expect("terminal for output"), + )) + .stderr(Stdio::from(terminal)); + command.spawn().expect("start the built shell") + // `command` ends here, and with it this process's handles on the + // shell's end of the terminal: reading `screen` stops once the + // shell has gone. + }; + + // Read on a thread of its own, so that a shell that never finishes + // cannot hold the test past `LIMIT`. + let (sender, sent) = mpsc::channel::>(); + let reader = std::thread::spawn(move || { + let mut chunk = [0u8; 4096]; + // Reading ends with an error once the shell has gone. + while let Ok(count) = screen.read(&mut chunk) { + if count == 0 || sender.send(chunk[..count].to_vec()).is_err() { + break; + } + } + }); + + let started = Instant::now(); + let mut to_type = lines.iter().copied().chain(["exit"]); + let mut transcript: Vec = Vec::new(); + let mut searched = 0; + let mut line_editor = false; + let mut typed_at = 0; + let status = loop { + let gone = match sent.recv_timeout(Duration::from_millis(20)) { + Ok(chunk) => { + transcript.extend_from_slice(&chunk); + false + } + Err(mpsc::RecvTimeoutError::Timeout) => false, + Err(mpsc::RecvTimeoutError::Disconnected) => true, + }; + + // The line editor asks for the cursor before it draws each prompt. + while let Some(at) = find(&transcript[searched..], CURSOR_QUERY) { + searched += at + CURSOR_QUERY.len(); + press(&mut keyboard, CURSOR_ANSWER); + if !line_editor { + // The first query means the editor is reading keys. Type + // every line now: it takes one line at each Enter and keeps + // the rest for its next prompt. + line_editor = true; + for line in to_type.by_ref() { + type_line(&mut keyboard, line); + } + } + } + + // The plain prompt asks nothing: it is printed, and the shell waits + // for a line. Type one line at each new prompt. + if !line_editor && transcript.len() > typed_at { + let shown = plain(&String::from_utf8_lossy(&transcript)); + if shown.ends_with(PROMPT_END) { + if let Some(line) = to_type.next() { + type_line(&mut keyboard, line); + typed_at = transcript.len(); + } + } + } + + if gone { + match shell.try_wait().expect("wait for the shell") { + Some(status) => break status, + // The terminal closes a moment before the shell can be + // waited for. + None => std::thread::sleep(Duration::from_millis(1)), + } + } + if started.elapsed() > LIMIT { + let _ = shell.kill(); + let _ = shell.wait(); + panic!( + "the shell did not finish within {LIMIT:?}; the terminal showed:\n{}", + plain(&String::from_utf8_lossy(&transcript)) + ); + } + }; + reader.join().expect("the reading thread"); + + let transcript = String::from_utf8_lossy(&transcript).into_owned(); + assert!( + to_type.next().is_none(), + "the shell left ({status}) before every line was typed; the terminal showed:\n{}", + plain(&transcript) + ); + transcript +} + +/// Sends `keys` to the shell as if they were pressed. +/// +/// The write fails only when the shell has gone. The transcript then says +/// why, and the assertions show it. +fn press(keyboard: &mut File, keys: &[u8]) { + let _ = keyboard.write_all(keys); +} + +/// Types `line` and presses Enter. +fn type_line(keyboard: &mut File, line: &str) { + press(keyboard, line.as_bytes()); + press(keyboard, b"\r"); +} + +fn find(haystack: &[u8], needle: &[u8]) -> Option { + haystack + .windows(needle.len()) + .position(|window| window == needle) +} + +/// `text` without the colour and cursor codes the line editor sends. +fn plain(text: &str) -> String { + let mut out = String::new(); + let mut chars = text.chars(); + while let Some(c) = chars.next() { + if c != '\x1b' { + out.push(c); + continue; + } + // Either `ESC [`, parameters and a final letter, or `ESC` and one + // character. + if chars.next() == Some('[') { + for c in chars.by_ref() { + if ('@'..='~').contains(&c) { + break; + } + } + } + } + out +} + +/// The lines the shell printed that start with `prefix`. +/// +/// A line the test typed never does: it is shown after the prompt. +fn printed(transcript: &str, prefix: &str) -> Vec { + plain(transcript) + .lines() + .map(str::trim_end) + .filter(|line| line.starts_with(prefix)) + .map(String::from) + .collect() +} + +/// A fresh directory holding `functions` in `defs.sh`, for `source`. +/// +/// The functions come from a file because the prompt cuts a typed line at +/// every `;`, also inside `{ }`: a body of two commands cannot be typed +/// there on one line. +fn directory_with(functions: &str) -> tempfile::TempDir { + let dir = tempfile::tempdir().expect("temporary directory"); + fs::write(dir.path().join("defs.sh"), functions).expect("write defs.sh"); + dir +} + +/// In every test the command that fails is `mkdir` under a directory that +/// does not exist. The built-in returns an error for it. +const NO_PARENT: &str = "mkdir /no/such/parent/dir"; + +#[test] +fn a_function_that_fails_leaves_the_callers_variable_as_it_was() { + let dir = directory_with(&format!("f() {{ local x=inner; {NO_PARENT}; }}\n")); + + let seen = at_the_prompt( + dir.path(), + &[ + "source ./defs.sh", + "x=outer", + "f", + r#"echo "after: x is $x.""#, + ], + ); + + assert_eq!( + printed(&seen, "after:"), + ["after: x is outer."], + "the terminal showed:\n{}", + plain(&seen) + ); + // The frame is popped; the failure is still reported. + assert_eq!( + printed(&seen, "vsh: Parent directory does not exist").len(), + 1, + "the failed mkdir should be reported once; the terminal showed:\n{}", + plain(&seen) + ); +} + +#[test] +fn a_function_that_fails_gives_the_caller_its_positional_parameters_back() { + let dir = directory_with(&format!("f() {{ {NO_PARENT}; }}\n")); + + let seen = at_the_prompt( + dir.path(), + &[ + "source ./defs.sh", + "set -- outer1 outer2", + "f inner1", + r#"echo "after: first is $1, count is $#.""#, + ], + ); + + assert_eq!( + printed(&seen, "after:"), + ["after: first is outer1, count is 2."], + "the terminal showed:\n{}", + plain(&seen) + ); +} + +/// `outer` calls `inner`, and a command in `inner` fails. The error passes +/// through both, so both frames have to go: the caller sees neither +/// `inner`'s variable and argument nor `outer`'s. +#[test] +fn a_failure_in_a_nested_call_pops_the_frame_of_every_function_it_leaves() { + let dir = directory_with(&format!( + "inner() {{ local x=in; {NO_PARENT}; }}\n\ + outer() {{ local x=mid; inner deep; }}\n" + )); + + let seen = at_the_prompt( + dir.path(), + &[ + "source ./defs.sh", + "x=top", + "set -- top1", + "outer mid1 mid2", + r#"echo "after: x is $x, first is $1, count is $#.""#, + ], + ); + + assert_eq!( + printed(&seen, "after:"), + ["after: x is top, first is top1, count is 1."], + "the terminal showed:\n{}", + plain(&seen) + ); +} + +/// The shell counts frames to know whether it is inside a function. With +/// the frame left behind, `local` and `return` typed at the prompt were +/// accepted. bash refuses both there. +#[test] +fn after_a_function_failed_the_shell_is_no_longer_inside_it() { + let dir = directory_with(&format!("f() {{ {NO_PARENT}; }}\n")); + + let seen = at_the_prompt( + dir.path(), + &["source ./defs.sh", "f", "local y=1", "return 0"], + ); + + for refused in ["vsh: local:", "vsh: return:"] { + assert_eq!( + printed(&seen, refused).len(), + 1, + "a line starting {refused:?} should be printed once; the terminal showed:\n{}", + plain(&seen) + ); + } +}