cat: resize the output pipe once per invocation by limadog9 · Pull Request #15506 · uutils/coreutils

limadog9 · 2026-10-18T19:29:42Z

Prepare stdout's pipe capacity once and reuse the broker copy for every input. On 30 files, resize attempts fall from 31 to 2. A strace regression checks one broker pipe and two resize attempts with cat /dev/null /dev/ null.

Closes #11700.

Sign in to view


 [target.'cfg(unix)'.dependencies]
-rustix = { workspace = true, features = ["fs"] }
+rustix = { workspace = true, features = ["fs", "pipe"] }


Sign in to view

+    // Increasing the output pipe capacity is optional and may be done once by
+    // callers that copy several inputs into the same output.
+    let _ = fcntl_setpipe_size(&mut *dest, MAX_ROOTLESS_PIPE_SIZE);
+    splice_unbounded_broker(source, dest)
+}
+
+/// Copy through a cached broker pipe without changing the output pipe capacity.
+///
+/// Callers copying multiple sources may prepare their output once and reuse
+/// this operation for each source. A failure to resize never prevents copying.
+#[inline]
+pub fn splice_unbounded_broker(source: &impl AsFd, dest: &mut impl AsFd) -> SpliceRes {


Sign in to view

-    // fcntl for input would not improve throughput since
-    // - sender with splice probably increased size already
-    // - sender without splice is bottleneck


Sign in to view

+        // All operands share stdout. Resize its pipe once, even if the kernel
+        // refuses the optional increase, rather than retrying for every file.


sylvestre · 2026-10-18T21:44:41Z

Sign in to view

    I: IntoIterator<Item = &'a OsString>,
 {
+    #[cfg(any(target_os = "linux", target_os = "android"))]
+    if options.can_print_fast() {


the description mentions a regression test for mixed file/stdin order but i don't see it in the commit, please add it in tests/by-util/test_cat.rs

If we test something, it should count pipe2 call by strace.
a regression checks mixed file/stdin output order is unrelated with this PR.

sylvestre · 2026-10-18T21:44:47Z

+/// Copy through a cached broker pipe without changing the output pipe capacity.
+///
+/// Callers copying multiple sources may prepare their output once and reuse
+/// this operation for each source. A failure to resize never prevents copying.


this function never resizes, so "a failure to resize never prevents copying" is confusing here. could you please make this doc one line?

I think we don't need this comment.

codspeed · 2026-10-19T08:31:05Z

Merging this PR will not alter performance

1 warning, 1 note

⚠️ 122 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

ℹ️ 1 benchmark has no flamegraph

The flamegraph wasn't generated: most of the time was spent in runtime internals, which would leave nothing useful to analyze.

See affected benchmarks

✅ 401 untouched benchmarks
⏩ 70 skipped benchmarks¹

_{Comparing limadog9:fix/cat-resize-output-pipe-once-11700 (e0daa92) with main (18bc31f)}

70 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

limadog9 · 2026-10-19T10:14:41Z

Added the requested strace regression

Sign in to view

 }
+
+#[test]
+fn test_fast_copy_mixed_file_and_stdin_order() {


Sign in to view

+
+    let scene = TestScenario::new(util_name!());
+    let probe = Command::new("strace")
+        .args(["-qq", "-o", "/dev/null", "-e", "trace=none"])


Sign in to view

+    let probe = Command::new("strace")
+        .args(["-qq", "-o", "/dev/null", "-e", "trace=none"])
+        .arg(&scene.bin_path)
+        .args(["cat", "--help"])


Sign in to view

 {
+    #[cfg(any(target_os = "linux", target_os = "android"))]
+    if options.can_print_fast() {
+        // All operands share stdout. Try to resize it once.


Sign in to view

+
+/// Copy through a cached broker pipe without resizing the output pipe.
+#[inline]
+pub fn splice_unbounded_broker(source: &impl AsFd, dest: &mut impl AsFd) -> SpliceRes {


oech3 · 2026-10-23T11:10:50Z

    static PIPE_CACHE: Cell<Option<(PipeReader, PipeWriter)>> = const { Cell::new(None) };
+    /// Cache the last destination's resize attempt, including failures.
+    /// Reused descriptor numbers may skip resizing, which is only an optimization.
+    static LAST_RESIZE_FD: Cell<Option<RawFd>> = const { Cell::new(None) };


Yes. This looks very reasonable and helpful also for other utils.

oech3 · 2026-10-23T11:15:39Z

+        eprintln!("TEST SKIPPED: strace is unavailable");
+        return;
+    };
+    let trace = read_to_string(scene.fixtures.plus("trace")).unwrap_or_default();


Suggested change

let trace = read_to_string(scene.fixtures.plus("trace")).unwrap_or_default();

let trace = read_to_string(scene.fixtures.plus("trace")).unwrap();

I think this should panic if it is missing.

oech3 · 2026-10-23T11:16:17Z

+    use std::process::Command;
+
+    let scene = TestScenario::new(util_name!());
+    let Ok(output) = Command::new("strace")


Suggested change

let Ok(output) = Command::new("strace")

let Ok(output) = std::process::Command::new("strace")

Command is used just once.

github-actions · 2026-10-23T12:36:27Z

Binary size comparison:

Individual binary size comparison VS main (threshold: >=5% AND >=4 KB).

Total size of compared binaries: 155.04 MB (-304 KB, -0.19%)

Significant per-binary changes:
  date     2.61 MB ->    2.30 MB  (-316 KB, -11.81%)

oech3 reviewed Oct 18, 2026

View reviewed changes

This comment was marked as low quality.

Sign in to view

oech3 reviewed Oct 18, 2026

View reviewed changes

Comment thread src/uucore/src/lib/features/pipes.rs

Comment on lines -112 to -114

// fcntl for input would not improve throughput since

// - sender with splice probably increased size already

// - sender without splice is bottleneck

This comment was marked as resolved.

Sign in to view

limadog9 force-pushed the fix/cat-resize-output-pipe-once-11700 branch from c4d75cf to 34525a4 Compare October 18, 2026 19:57

oech3 reviewed Oct 18, 2026

View reviewed changes

Comment thread src/uu/cat/src/cat.rs Outdated

Comment on lines +415 to +416

// All operands share stdout. Resize its pipe once, even if the kernel

// refuses the optional increase, rather than retrying for every file.

This comment was marked as outdated.

Sign in to view

limadog9 force-pushed the fix/cat-resize-output-pipe-once-11700 branch from 34525a4 to c7f35d0 Compare October 18, 2026 20:40

sylvestre reviewed Oct 18, 2026

View reviewed changes

limadog9 force-pushed the fix/cat-resize-output-pipe-once-11700 branch from c7f35d0 to 736f4f1 Compare October 18, 2026 22:29

limadog9 force-pushed the fix/cat-resize-output-pipe-once-11700 branch from 736f4f1 to adfcce3 Compare October 19, 2026 10:06

oech3 reviewed Oct 19, 2026

View reviewed changes

Comment thread tests/by-util/test_cat.rs Outdated

}

#[test]

fn test_fast_copy_mixed_file_and_stdin_order() {

This comment was marked as resolved.

Sign in to view

oech3 reviewed Oct 19, 2026

View reviewed changes

Comment thread tests/by-util/test_cat.rs Outdated

let scene = TestScenario::new(util_name!());

let probe = Command::new("strace")

.args(["-qq", "-o", "/dev/null", "-e", "trace=none"])

This comment was marked as resolved.

Sign in to view

oech3 reviewed Oct 19, 2026

View reviewed changes

Comment thread tests/by-util/test_cat.rs Outdated

let probe = Command::new("strace")

.args(["-qq", "-o", "/dev/null", "-e", "trace=none"])

.arg(&scene.bin_path)

.args(["cat", "--help"])

This comment was marked as outdated.

Sign in to view

This comment was marked as resolved.

Sign in to view

limadog9 force-pushed the fix/cat-resize-output-pipe-once-11700 branch from adfcce3 to 74bd78d Compare October 19, 2026 10:43

oech3 reviewed Oct 23, 2026

View reviewed changes

Comment thread src/uu/cat/src/cat.rs Outdated

{

#[cfg(any(target_os = "linux", target_os = "android"))]

if options.can_print_fast() {

// All operands share stdout. Try to resize it once.

This comment was marked as resolved.

Sign in to view

oech3 reviewed Oct 23, 2026

View reviewed changes

Comment thread src/uucore/src/lib/features/pipes.rs Outdated

/// Copy through a cached broker pipe without resizing the output pipe.

#[inline]

pub fn splice_unbounded_broker(source: &impl AsFd, dest: &mut impl AsFd) -> SpliceRes {

This comment was marked as resolved.

Sign in to view

uucore: cache output pipe resize attempts

e0daa92

limadog9 force-pushed the fix/cat-resize-output-pipe-once-11700 branch from 74bd78d to e0daa92 Compare October 23, 2026 10:54

oech3 reviewed Oct 23, 2026

View reviewed changes

		// All operands share stdout. Resize its pipe once, even if the kernel
		// refuses the optional increase, rather than retrying for every file.

	let trace = read_to_string(scene.fixtures.plus("trace")).unwrap_or_default();
	let trace = read_to_string(scene.fixtures.plus("trace")).unwrap();

	let Ok(output) = Command::new("strace")
	let Ok(output) = std::process::Command::new("strace")

Conversation

limadog9 commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as low quality.

Uh oh!

This comment was marked as low quality.

Uh oh!

This comment was marked as low quality.

Uh oh!

This comment was marked as low quality.

This comment was marked as resolved.

Uh oh!

This comment was marked as outdated.

Uh oh!

sylvestre Oct 18, 2026

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

oech3 Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

sylvestre Oct 18, 2026

Choose a reason for hiding this comment

Uh oh!

oech3 Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codspeed Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

limadog9 commented Oct 19, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

oech3 Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

oech3 Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

oech3 Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Oct 23, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

limadog9 commented Oct 18, 2026 •

edited

Loading

oech3 Oct 18, 2026 •

edited

Loading

codspeed Bot commented Oct 19, 2026 •

edited

Loading