dd: stop faulting in the whole copy buffer before reading

[luantaraschi]

dd reserves the copy buffer and then fills it with BUF_INIT_BYTE, so every
page of bs= is written before the first read, whether or not there is anything
to copy. GNU dd leaves the buffer alone.

Measured against GNU coreutils 9.7 in a container with 15 GiB of RAM, debug build:

command	GNU	before	after
bs=1G count=1 iflag=nonblock if=<empty fifo> of=/dev/null	1.3 MiB, 0.00 s	1029 MiB, 3.00 s	5.2 MiB, 0.02 s
obs=11777777 ibs=1111 bs=7177118117 cbs=8818181111 if=/dev/null of=a	1.5 MiB, 0.00 s	6.7 GiB, 19.48 s	5.5 MiB, 0.02 s

Read::read does need an initialised slice, but zeroed pages are free.
vec![0; n] allocates through alloc_zeroed, so the pages come from the kernel
already zero and are never touched until something is read into them. The
reservation still runs first, because vec![0; n] aborts when the allocation
fails and dd has to report an error instead. The existing bs=1PB tests cover
that path.

The attempts in #11555 and #11577 went after reading into uninitialised memory,
which needs unsafe or an unstable Read::read_buf. This does not. The buffer
stays zeroed, it just never gets written to.

One case is left over. After a short read the buffer is truncated and grown
again on the next iteration, and growing it writes over the new region, so
bs=2G on a ten byte file still peaks at 2 GiB. Neither reported case reaches
it, since both read nothing, but removing the truncate and regrow cycle is a
separate change.

#12143 and #13373 rework the same allocation for O_DIRECT alignment. Both keep a
resize, so the fault-in would survive either of them.

Fixes #13869

Refs #11544. The write there is the resize in read_helper, and this only
makes that call a no-op while the buffer still has its full length, so that one
gets better here without being closed.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[oech3]

Why reallocating at here? It causes OOM race.

[luantaraschi]

You are right that it allocates twice. The first one is there for the failure path: vec![0; n] goes through handle_alloc_error, so with nothing fallible in front of it dd bs=1PB aborts instead of reporting an error.

I built it both ways to be sure. With the reservation: dd: IO error: out of memory, exit 1. Without it: memory allocation of 1000000000000000 bytes failed, SIGABRT, exit 134. That second one is what test_huge_obs_reports_memory_error_instead_of_aborting was added for, in #12847.

The window you are pointing at is real, though. Between the drop and the second allocation another allocation can take the space, and then this one aborts anyway. I could not find a fallible alloc_zeroed in safe stable Rust, and Vec::try_with_capacity is still open as rust-lang/rust#91913, so the way out I can see is a single alloc_zeroed behind a small unsafe block, which drops the second allocation and the window at once. dd.rs already has one unsafe block, at File::from_raw_fd. Would you rather see it that way?

[oech3]

This project does not accept unsafe just for performance. So we should improvement for std.

[oech3]

I don't think this change avoids filling by 0 and different with previous code.

[oech3]

Please don't link #11544 as closed by this PR which is coming from different fn.

[luantaraschi]

On the zeros: they are still there, that part does not change. What changes is who writes them. vec![0; n] goes through alloc_zeroed, so at this size the pages come back from the kernel already zero and nothing touches them until a read lands in them. resize memsets the whole thing up front.

The command from #11544, rebuilt today, debug build, peak RSS from /usr/bin/time -v, three runs each:

GNU 9.7     1.5 MiB    0.00 s
main       1030 MiB    2.7 s
this PR     5.4 MiB    0.02 s

On the link, you are right and I have dropped it. The write is the buf.resize(bsize, BUF_INIT_BYTE) in read_helper, and all this does is make that call a no-op while the buffer still has its full length. After a short read the buffer is truncated and grown again and the fill comes back, so #11544 is not gone in general and should not close from here. I kept Fixes #13869, where the read returns zero on the first pass and there is no regrow: the literal command from that issue runs at 5.6 MiB and 0.02 s here.