Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions CHANGELOG.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

== [Unreleased]

=== Fixed — 2026-11-18 (arguments: globs, quote removal)

- Glob expansion now applies to the arguments of every external command,
every pipeline stage, `echo` and function calls. Before, only an external
command that also had a redirection expanded its arguments: `wc -c *.txt`
passed `*.txt` to `wc`, `ls *.txt | wc -l` and `echo *.txt` did not expand.
- Quoted arguments no longer reach the program with a backslash in front of
`*`, `?`, `[` or `{`: `echo '*.txt'` printed `\*.txt` and now prints
`*.txt`. A partly quoted word globs only its unquoted part (`'*'x.*`).
- `echo` performs command substitution and arithmetic expansion
(`echo $((2+3))` printed `$((2+3))`), and `$(echo ...)` works.
- `$((1+(2*3)))` and other substitutions containing `(` are read to their
matching `)`.
- An expansion error (such as a glob limit from #191) fails the command with
exit status 1 and does not run it. With a redirection the status was 127.

=== Changed — 2026-11-18 (typed words)

- New module `src/word.rs`: a `Word` keeps the quoting of each of its pieces
(unquoted, single, double, backslash), and `word::expand_words` performs
tilde, parameter, command, arithmetic and process substitution, brace
expansion, glob expansion and quote removal on it. `Command::External`,
`Command::Pipeline` and `Command::Echo` now carry `Vec<Word>` arguments.
Other commands still use the older string encoding; see
`docs/POSIX_COMPLIANCE.adoc` (Milestones 5 and 6) for what is not migrated.
- Removed `external::expand_glob_args`; its rules (no match or invalid
pattern keeps the word, a limit is an error) moved to `word.rs` with their
tests.

=== Fixed — 2026-11-18 (glob expansion out of memory, #191)

- `**` no longer follows symbolic links to directories, so a link back to a
Expand Down
12 changes: 12 additions & 0 deletions docs/POSIX_COMPLIANCE.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,12 @@ or literal result (#191, 2026-11-18)
|Brace expansion `+{a,b,c}+` |Done |Nested braces supported

|Sorted results |Done |POSIX requirement

|Where globs expand |Partial |Arguments of external commands (with or
without redirections), every pipeline stage, `+echo+` and function calls,
through `+word::expand_words+` (2026-11-18; before, only external commands
with a redirection). Not yet: builtin operands (`+ls+`, `+rm+`, `+cp+`, ...),
`+for+` word lists, redirection targets
|===

==== Milestone 6: Quote Processing — COMPLETE
Expand All @@ -194,6 +200,12 @@ quotes
|Unclosed quote detection |Done |

|Line continuation |Done |

|Quote removal |Partial |Done for the arguments that go through
`+word::expand_words+` (see Milestone 5), which keeps the quoting of each
piece of a word. Builtin operands, redirection targets, assignment values,
`+case+` patterns and `+test+` arguments still receive quoted `+*?[{~+` with
a backslash in front (`+touch '*q'+` creates `+\*q+`)
|===

==== Milestone 7: Command Substitution — COMPLETE
Expand Down
2 changes: 1 addition & 1 deletion impl/rust-cli/src/commands.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2856,7 +2856,7 @@ mod tests {
let state = ShellState::new(tmp.path().to_str().unwrap()).unwrap();
let cmd = crate::parser::Command::External {
program: "git".to_string(),
args: vec!["status".to_string()],
args: vec!["status".into()],
redirects: vec![],
background: false,
};
Expand Down
121 changes: 68 additions & 53 deletions impl/rust-cli/src/executable.rs
Original file line number Diff line number Diff line change
Expand Up @@ -488,35 +488,36 @@ impl ExecutableCommand for Command {
// Expand program name (no process substitutions in program name)
let expanded_program = crate::parser::expand_with_command_sub(program, state)?;

// Word expansion: variables, substitutions, braces, globs and
// quote removal. A failed expansion (for example a glob that
// reaches a limit) fails the command without running it.
let expansion =
match crate::word::expand_words(args, state, crate::word::ProcessSubs::Allow) {
Ok(expansion) => expansion,
Err(e) => {
eprintln!("vsh: {}: {}", expanded_program, e);
return Ok(ExecutionResult::ExternalCommand { exit_code: 1 });
}
};

// Check if the command is a shell function before external execution.
// `if let Some(...)` performs a single map lookup; the former
// double-lookup plus expect shape could panic and duplicated
// the lookup.
if let Some(func_def) = state.functions.get(&expanded_program).cloned() {
let expanded_args: Vec<String> = args
.iter()
.map(|a| crate::parser::expand_variables(a, state))
.collect();
return execute_function_call(&func_def, &expanded_args, state);
let result = execute_function_call(&func_def, &expansion.fields, state);
expansion.cleanup();
return result;
}

// Expand arguments with process substitutions
let mut all_process_subs = Vec::new();
let mut expanded_args = Vec::new();

for arg in args {
let (expanded_arg, proc_subs) =
crate::parser::expand_with_process_sub(arg, state)?;
expanded_args.push(expanded_arg);
all_process_subs.extend(proc_subs);
}
let expanded_args = &expansion.fields;

// Execute main command
if *background {
// Background execution
let _job_id = external::execute_external_background(
&expanded_program,
&expanded_args,
expanded_args,
redirects,
state,
)
Expand All @@ -525,24 +526,19 @@ impl ExecutableCommand for Command {
0 // No job created
});

// Cleanup all process substitutions
for proc_sub in all_process_subs {
if let Err(e) = proc_sub.cleanup() {
eprintln!("Warning: Failed to cleanup process substitution: {}", e);
}
}
expansion.cleanup();

Ok(ExecutionResult::Success)
} else {
// Foreground execution
let exit_code = if redirects.is_empty() {
// No redirections - use simple path
external::execute_external(&expanded_program, &expanded_args)
external::execute_external(&expanded_program, expanded_args)
} else {
// Has redirections - use redirection-aware execution
external::execute_external_with_redirects(
&expanded_program,
&expanded_args,
expanded_args,
redirects,
state,
)
Expand All @@ -552,12 +548,7 @@ impl ExecutableCommand for Command {
127
});

// Cleanup all process substitutions
for proc_sub in all_process_subs {
if let Err(e) = proc_sub.cleanup() {
eprintln!("Warning: Failed to cleanup process substitution: {}", e);
}
}
expansion.cleanup();

Ok(ExecutionResult::ExternalCommand { exit_code })
}
Expand All @@ -569,33 +560,42 @@ impl ExecutableCommand for Command {
redirects,
background,
} => {
// Expand variables and command substitutions in all pipeline stages
let expanded_stages: Result<Vec<(String, Vec<String>)>> = stages
.iter()
.map(|(program, args)| {
let expanded_program =
crate::parser::expand_with_command_sub(program, state)?;
let expanded_args: Result<Vec<String>> = args
.iter()
.map(|arg| crate::parser::expand_with_command_sub(arg, state))
.collect();
Ok((expanded_program, expanded_args?))
})
.collect();
let expanded_stages = expanded_stages?;

if *background && !expanded_stages.is_empty() {
if *background && !stages.is_empty() {
return Err(anyhow::anyhow!(
"Background pipelines are not supported yet; refusing to run them in the foreground"
));
}

// Expand every stage before any stage runs, so a failed
// expansion runs no part of the pipeline.
let mut expanded_stages: Vec<(String, Vec<String>)> = Vec::new();
let mut process_subs = Vec::new();
for (program, args) in stages {
let expanded_program = crate::parser::expand_with_command_sub(program, state)?;
match crate::word::expand_words(args, state, crate::word::ProcessSubs::Allow) {
Ok(expansion) => {
expanded_stages.push((expanded_program, expansion.fields));
process_subs.extend(expansion.process_subs);
}
Err(e) => {
eprintln!("vsh: {}: {}", expanded_program, e);
return Ok(ExecutionResult::ExternalCommand { exit_code: 1 });
}
}
}

let exit_code = external::execute_pipeline(&expanded_stages, redirects, state)
.unwrap_or_else(|e| {
eprintln!("Pipeline error: {}", e);
127
});

crate::word::Expansion {
fields: Vec::new(),
process_subs,
}
.cleanup();

Ok(ExecutionResult::ExternalCommand { exit_code })
}

Expand Down Expand Up @@ -684,10 +684,15 @@ impl ExecutableCommand for Command {
interpret_escapes,
redirects,
} => {
let expanded_args: Vec<String> = args
.iter()
.map(|a| crate::parser::expand_variables(a, state))
.collect();
let expansion =
match crate::word::expand_words(args, state, crate::word::ProcessSubs::Allow) {
Ok(expansion) => expansion,
Err(e) => {
eprintln!("vsh: echo: {}", e);
return Ok(ExecutionResult::ExternalCommand { exit_code: 1 });
}
};
let expanded_args = &expansion.fields;

let output = if *interpret_escapes {
expanded_args
Expand Down Expand Up @@ -719,6 +724,7 @@ impl ExecutableCommand for Command {
Ok(())
})?;
}
expansion.cleanup();
Ok(ExecutionResult::Success)
}

Expand Down Expand Up @@ -1291,7 +1297,7 @@ impl ExecutableCommand for Command {
if args.is_empty() {
program.clone()
} else {
format!("{} {}", program, args.join(" "))
format!("{} {}", program, join_words(args))
}
}
Command::Pipeline { stages, .. } => {
Expand All @@ -1301,7 +1307,7 @@ impl ExecutableCommand for Command {
if args.is_empty() {
prog.clone()
} else {
format!("{} {}", prog, args.join(" "))
format!("{} {}", prog, join_words(args))
}
})
.collect();
Expand Down Expand Up @@ -1377,7 +1383,7 @@ impl ExecutableCommand for Command {
format!("{} {} {}", left.description(), op_str, right.description())
}

Command::Echo { args, .. } => format!("echo {}", args.join(" ")),
Command::Echo { args, .. } => format!("echo {}", join_words(args)),
Command::True => "true".to_string(),
Command::False => "false".to_string(),
Command::Read { var_names, .. } => format!("read {}", var_names.join(" ")),
Expand Down Expand Up @@ -1645,6 +1651,15 @@ fn execute_loop(
Ok(last_result)
}

/// Words written back as source and separated by spaces.
fn join_words(words: &[crate::word::Word]) -> String {
words
.iter()
.map(|w| w.to_string())
.collect::<Vec<_>>()
.join(" ")
}

/// Match a word against a case pattern (supports glob-style wildcards)
fn case_pattern_matches(word: &str, pattern: &str) -> bool {
if pattern == "*" {
Expand Down
Loading
Loading