std::io::copy's splice(2) optimization does not use extended pipe on Linux

[oech3]

View all comments

uutils/coreutils head -c has splice fast-path which is not using io::copy's internal splice and manually calling it with 1 MiB pipe size:

$ truncate -s 1PB /tmp/huge
$ head -c 1PB /tmp/huge|pv>/dev/null
^C.3GiB 0:00:01 [37.3GiB/s]

I removed cfg for Linux from
https://github.com/uutils/coreutils/blob/6b16cc9688fe998f5dc2786f53f22ceea709e318/src/uu/head/src/head.rs#L169-L177 and tried

fn print_n_bytes(input: impl Read, n: u64) -> io::Result<u64> {
    // Read the first `n` bytes from the `input` reader.
    let mut reader = input.take(n);

    // Write those bytes to `stdout`.
    let stdout = io::stdout();
    // larger pipe size should improve throughtput with splice
    let _ = rustix::pipe::fcntl_setpipe_size(&stdout, 1024*1024);
    let mut stdout = stdout.lock();

    let bytes_written = io::copy(&mut reader, &mut stdout).map_err(wrap_in_stdout_error)?;

Then I have

$ target/debug/head -c 1PB /tmp/huge|pv>/dev/null
^C21GiB 0:00:02 [2.10GiB/s]

I think std::io::copy should honor (or implicitly extend pipe size of target to 1 MiB).

[xtqqczze]

@oech3 Could you verify whether std::io::copy is actually using splice syscalls in your testing? It may be preferring sendfile instead.

[oech3]

it was using splice
(sendfile is implemented using splice internally at recient Linux.)

[xtqqczze]

let _ = rustix::pipe::fcntl_setpipe_size(&stdout, 1024*1024);

Is this operation (which returns a Result) actually succeeding?

[oech3]

yes. I guess std is hardcoding default pipe size or something else as arg of splice.

[oech3]

Strange, it is much larger than 1MiB

rust/library/std/src/sys/io/kernel_copy/linux.rs

Line 760 in ba0949a

let chunk_size = crate::cmp::min(len - written, 0x7ffff000_u64) as usize;

@the8472 Is it able to change this size to one returned by fcntl_setpipe_size?

[the8472]

Check with strace if it's even using splice. You might running into a case where we disabled splicing for data-consistency reasons (#108283).

[oech3]

Yes. I saw splice with strace #155859 (comment)

[the8472]

Well, you're using pv there which also splices, so you need to make sure you're looking at the right process.
Can you post some strace excerpt?