Why is dd reading from and writing to the same disk?

Question

I've been using dd in Linux to overwrite an external USB hard drive.

When I use the default block size of 512 bytes, with this command:

# dd if=/dev/zero of=/dev/sdb

throughout the whole operation, the hard drive (/dev/sdb) is being read from and written to alternately, approximately 1GB at a time. I.e. read 1GB...write 1GB...read 1GB...write 1GB etc. As much data is read from the hard drive as is written to it.

I know this is happening because it's showing in my custom Conky panel (diskio_read, diskio_write), which I know to be a 100% reliable indicator of disk I/O activity.

I've repeated this using a different external hard drive on a different computer. It happens via both USB 2.0 and USB 3.0.

In contrast, when I do the same thing, but use a block size of 1MB instead, with this command:

# dd if=/dev/zero of=/dev/sdb bs=1M

apart from a small amount of reading at the start, the hard drive is not read from at all during the operation.

Given that this phenomenon has happened on two different computers and two different hard drives of mine, using a standard Linux distro (Xubuntu 14.04), anyone who wants to should presumably be able to replicate it on their own computer.

Can someone please explain what is happening here?

Where/how do you observe those read/write? (iostat, strace, btrace)? I assume it's Linux, right? What does grep . /sys/block/sdb/queue/*size* give? — Stéphane Chazelas, Dec 03 '15 at 17:16
I've elaborated on how I know about the I/O. I can't read that file because I'm not overwriting the hard drive any more. — EmmaV, Dec 03 '15 at 17:32
@Gilles My intention was to write nulls through dm-crypt, so I expect there will be the same problem if I use cat. — EmmaV, Dec 06 '15 at 00:28

score 7 · Accepted Answer · answered Dec 03 '15 at 17:58

If you specify a block size (512 bytes) of less than the block size of the disk (often 4096 bytes, but nowadays maybe more), the block will be partially written, so that the contents of the rest of the block must be preserved before writing.

This is because disk blocks cannot be written to with only 512 bytes, but you have to write a full block at once (4096 or larger). When you write this (4096) amount or more, there is no partial write, so it does not have to read.

score 0 · Answer 2 · edited Apr 13 '17 at 12:36

0

This is basically a symptom of buffered I/O on a Linux system. A good overview of this in relation to the dd command is located at Is there a way to determine the optimal value for the bs parameter to dd? which supports ctrl-d's answer. However, this also suggests that even larger block sizes (i.e., greater than 1M) might improve performance further.

edited Apr 13 '17 at 12:36

Community

1

answered Dec 03 '15 at 18:37

Thomas N

744

Actually, the reads are not due to buffered I/O. They would be just as necessary if I/O was unbuffered. – Gilles 'SO- stop being evil' Dec 03 '15 at 23:46

Why is dd reading from and writing to the same disk?

2 Answers2