uucore: share GNU argument parsing across cat, rm and ls

[limadog9]

Summary

Implemented proposal for architectural review: add a shared GNU-oriented argument scanner in uucore and use it for normal runtime parsing in cat, rm, ls, dir and vdir. Ordinary parsing constructs neither a Clap command nor ArgMatches. Clap remains for requested help/version presentation, uu_app(), documentation, completions and existing tooling interfaces.

The strict differential gate still exits 1 on 201 changed diagnostic/interface outcomes requiring review. This draft seeks design feedback, not immediate merge.

Motivation

GNU command lines require ordered option precedence, clustered short options, required values resembling options, and optional-value attachment rules. POSIXLY_CORRECT, including an empty value, can stop parsing at the first operand. Resolving these through general-purpose Clap matches requires utility-specific workarounds. Separating scanning also avoids constructing translated help metadata on ordinary successful paths.

Related: #14621, #4254 and #9103. This migrates five entry points; it does not completely resolve those issues or eliminate localization startup costs.

Architecture

• Static per-utility option schemas define identities, spellings, aliases and arity, with lazy presentation metadata projected into Clap.
• The scanner returns ordered occurrences with argument positions and raw OsString operands under an explicit ordering policy. Typed utility settings retain precedence and environment/default behavior.
• Existing public interfaces remain, including uu_app() and the ls Config::from/from_dir/from_vdir Clap adapters sharing typed settings logic.
• Scanning, settings and presentation are separate from execution. No deletion/traversal algorithm changes, new crate or dependency are introduced. Common lexical rules are reusable while utility-specific interactions stay local.

Validation

Recorded development-environment execution, not independent maintainer verification:

• Three pinned GNU 9.12 differential runs, 1,516 cases each: seeds 4254/1729/20261017 report 95/96/94 exact compatibility improvements and zero cases classified as new regressions. Their gates remain failing as detailed below.
• 108 uucore tests (25 parser), 38 utility unit tests, 409/401 enabled integration tests with/without diagnostics, 905 public-adapter comparisons, and 63 Python CI tests (29 checker). Root-protection tests ran in disposable confinement; three existing integration ignores remain.
• 492 tooling/localization executions; tested help, version, manual and completion output stayed byte-for-byte unchanged. English/French/fallback checks passed. Existing GNU workflow: 47 passed, one missing-gdb skip.
• MSRV 1.88, SMACK and Windows compilation checks passed; Windows binaries were not executed. Parser tests passed on native musl, emulated i686 and WASI. Formatting, Clippy, Rustdoc, Ruff and spelling checks passed.

There is no new full-feature release or SELinux validation claim; required host SELinux headers were unavailable. Build configurations and verification limits.

Behavioral differences requiring review

All three strict gates exit 1: 201 changed diagnostic/interface outcomes remain unapproved. All 211 initially flagged cases were investigated and mapped: two exposed a real rm dash-filename-advice regression, now fixed; 115 reflect GNU corrections with another mismatch remaining; 84 require diagnostic/extension decisions; ten POSIX informational-request cases were checker misclassifications and now classify as exact GNU behavioral fixes.

Remaining differences include diagnostic layout, short-token identity, alias/choice lists, raw-byte quoting, multicall hints and an inherited hidden-alias collision. They are not treated as passing tests; no allowlist was added.

Evidence branch · Complete 211-case reconciliation with exact results · Full differential summaries. Evidence is on a separate branch and excluded from this diff.

Performance and binary size

Recorded comparisons against original base dd74d76072c69ae9ef1effb50599d49d6f360898, using Rust 1.99.1 release, fat LTO, one codegen unit and panic-abort: the selected-eight multicall build (cat,rm,ls,dir,vdir,true,basename,wc,feat_diagnostics) decreased from 4,443,240 to 4,405,744 bytes (0.84%). Matching-profile standalone ls decreased from 3,936,272 to 3,730,408 bytes (5.23%); cat/rm/dir/vdir also decreased. Standalone builds enabled uucore/diagnostics.

Latest rm and true/basename/wc control timings overlap in variation: no general runtime speedup or allocation reduction is established. These are selected-feature/original-base comparisons, not full-feature or current-main benchmarks.

Design questions for maintainers

1. Is a shared GNU-specific runtime argument parser desirable in uucore?
2. Is the scanner/settings/Clap-presentation separation acceptable long term?
3. Should this remain one PR or be reviewed through smaller stacked PRs?
4. Which diagnostic compatibility differences must be resolved before merging?

Related work and provenance

The archived uutils-args effort and #4254 are prior design context, not approval of this migration. Recorded provenance states no archived uutils-args implementation code was reused and no GNU implementation source was copied.

[codspeed]

Merging this PR will regress 1 benchmark

⚠️ 119 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

• mv_force_overwrite
• rm_single_file
• cat_default[10000]
• cat_default[10000000]
• rm_force_files
• rm_multiple_files
• ls_recursive_mixed_tree
• ls_recursive_deep_tree[(200, 2)]
• ls_recursive_long_all_mixed_tree
• ls_recursive_long_all_deep_tree[(100, 4)]
• …and 109 more

ℹ️ 1 benchmark has no flame graph

The flamegraph could not be generated because too much time was spent in runtime internals, making the visualization not useful for performance analysis.

• sort_dictionary_order[500000]

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 22 improved benchmarks
❌ 1 regressed benchmark
✅ 378 untouched benchmarks
⏩ 70 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌ ⚠️	Simulation	mv_force_overwrite	147.4 ms	168.9 ms	-12.69%
⚡	Memory	cat_default[10000000]	19,924 B	173 B	×120
⚡	Memory	cat_default[10000]	87.5 KB	1.2 KB	×76
⚡ ⚠️	Simulation	rm_single_file	104.1 ms	4.1 ms	×25
⚡ ⚠️	Simulation	cat_default[10000]	420.1 µs	26.8 µs	×16
⚡ ⚠️	Simulation	cat_default[10000000]	179.9 µs	21.4 µs	×8.4
⚡	Memory	rm_recursive_tree	133.2 KB	40.4 KB	×3.3
⚡	Memory	rm_force_files	281.2 KB	117.6 KB	×2.4
⚡	Memory	rm_multiple_files	280.6 KB	117.6 KB	×2.4
⚡ ⚠️	Simulation	rm_force_files	4.6 ms	2.4 ms	+89.19%
⚡ ⚠️	Simulation	rm_multiple_files	4.8 ms	2.8 ms	+73.6%
⚡	Memory	ls_recursive_long_all_balanced_tree[(6, 4, 15)]	321.3 KB	207.6 KB	+54.79%
⚡	Memory	ls_recursive_balanced_tree[(6, 4, 15)]	111.1 KB	82.2 KB	+35.13%
⚡	Memory	ls_recursive_long_all_deep_tree[(100, 4)]	117 KB	88.1 KB	+32.78%
⚡	Memory	ls_recursive_mixed_tree	117.9 KB	89.1 KB	+32.43%
⚡	Memory	ls_recursive_long_all_mixed_tree	118.4 KB	89.5 KB	+32.26%
⚡ ⚠️	Simulation	ls_recursive_mixed_tree	2.7 ms	2.1 ms	+30.62%
⚡	Memory	ls_recursive_deep_tree[(200, 2)]	125 KB	96.1 KB	+30.06%
⚡ ⚠️	Simulation	ls_recursive_deep_tree[(200, 2)]	3.4 ms	2.7 ms	+26.06%
⚡ ⚠️	Simulation	ls_recursive_long_all_mixed_tree	5.1 ms	4.3 ms	+17.95%
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing limadog9:feature/gnu-parser-ls-cat-rm (e7d9baa) with main (5fda18e)²}

Footnotes

1. 70 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on main (f944de0) during the generation of this report, so 5fda18e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[github-actions]

Binary size comparison:

Individual binary size comparison VS main (threshold: >=5% AND >=4 KB).

Total size of compared binaries: 154.67 MB (-736 KB, -0.46%)

Significant per-binary changes:
  ls       3.27 MB ->    3.08 MB  (-192 KB, -5.73%)
  dir      2.12 MB ->    1.94 MB  (-188 KB, -8.64%)
  vdir     2.12 MB ->    1.94 MB  (-188 KB, -8.64%)
  rm       1.32 MB ->    1.19 MB  (-136 KB, -10.06%)
  cat      1.12 MB ->    1016 KB  (-132 KB, -11.50%)

[limadog9]

• Removed both Python files and the custom CI changes.
• Added coverage in the existing Rust suites.
• Corrected the macOS filename-fixture guards.

[oech3]

How many diff do we need if we applied this change for all utils.
Also we would not save multi-call binary's size until we remove clap from all utils.

[limadog9]

A full conversion would be substantial. I haven’t audited every utility, so I don’t have a reliable project-wide diff estimate yet.

This PR has three runtime parser integrations: cat, rm and ls. dir/vdir reuse the ls implementation. The shared scanner/schema module is roughly 780 lines before its test module, including comments and blank lines; that code is added once, not copied per utility.

For concrete examples, the current implementation-file diffs are cat.rs +154/-125 and rm.rs +299/-206, separate from their integration tests. ls is considerably larger because of its option interactions and public Config adapters. Each additional migration would still need its option schema, settings mapping and compatibility tests; it would not be a mechanical replacement of the Clap call.

On multicall size, agreed. This approach does not eliminate Clap. Even after migrating the runtime parsers, Clap would remain for help, documentation and existing interfaces. The standalone binary reductions should not be extrapolated to the full multicall binary.

The primary motivation is GNU argument compatibility and avoiding Clap command construction during ordinary execution, rather than eliminating Clap as a dependency. The shared scanner is intended to make those parsing rules reusable without duplicating the implementation across utilities.

The tradeoff is maintaining that scanner alongside Clap’s presentation layer. This PR provides three concrete integrations to evaluate that cost and benefit before considering any broader migration.