Check dd's progress without USR1?

Question

I'm on a Linux busybox 1.27 only system so no output=progress available, no busybox's own implementation of pv which is pipe_progress nor pv itself.

I have two questions. The first is based on https://www.linux.com/training-tutorials/show-progress-when-using-dd/. It says that by sending the USR1 signal to dd it "pauses" the process and dd after printing its current status will continue with the job it was doing. I'm trying to do some benchmark tests with dd so I would like to have minimal impact on the dd operation. I want to get an output of the current operation every second because the data that's passing through dd is fluctuating and it is important to me to recognize when the transfer rate drops.

First question: Is it true that 'dd' "pauses" every time it receives a USR1 signal?

If dd pauses every second then I'll be adding hours to the operation when tens of gigabytes are being transferred.

Second question: Assuming yes as an answer to the first question, I would like to know if it's possible to get dd to print its current status without sending any signal to the process, maybe some kind of redirection for STDOUT (like 2>&1)?

What I'm referring to is:

# bs with 1Mib so I can have more control on the test.
dd if=/dev/zero of=/dev/null bs=1048576 count=1024
Printing current operation status.
sudo kill -USR1 $dd_pid 

Answer 1

dd if=/dev/zero of=/dev/null bs=1048576 count=1024

Note that dd can mangle data, at least when using the bs parameter. And its performance advantage is small at best if you hand-pick an optimal block size for your particular system configuration: cat or cp can do better, and at most is only slightly slower. So don't use dd if you don't have to.

Note that since version 1.23, BusyBox uses the sendfile system call to copy data, instead of using read and write. Only plain copies such as cat and cp use sendfile, however: dd is forced to use read/write because it wants precise control over the sizes. So with BusyBox ≥1.23, cat and cp are very likely to be faster than dd.

Is it true that 'dd' "pauses" every time it receives an USR1 signal?

Technically yes, it has to “pause” to handle the signal. However the pause is only a few CPU instructions (the most costly part by far is printing the progress output). So this doesn't invalidate your benchmark in any way.

If dd pauses every second then I'll be adding hours to the operation when tens of gigabytes are being transferred.

No, you have your orders of magnitude wrong. You'll be adding maybe .1% of the time on a single-CPU thread. The main cost is kernel time for the benchmarking program, not dd, so it's intrinsic in what you want to do, not in the way you implement it.

if its possible to get dd to print its current status without sending any signal to the process

Well, no. There's already a simple, historically established, standard, easy way to do it. Why would there be another way which would be harder to implement?

---

On Linux, there is a generic way to know the point that a copy has reached. It doesn't depend on what program is doing the copy, though it doesn't always work with special files. Find out the process ID $pid that's doing the copy, and which file descriptors it's using for input and output. dd reads from fd 0 and writes to fd 1. BusyBox cp typically reads from fd 3 and writes to fd 4. You can check which file is open on which file descriptor through the symbolic links in /proc/$pid/fd.

$ cp /dev/zero /dev/null & pid=$!
$ readlink /proc/$pid/fd/3
/dev/zero
$ readlink /proc/$pid/fd/4
/dev/null

And you can check the position on the file descriptor $n$ in /proc/$pid/fd/$n.

$ cat /proc/$pid/fdinfo/4
pos:    74252288
flags:  0100001
mnt_id: 27

However, note that the file descriptor position may not updated with special files such as /dev/zero, /dev/null, pipes or sockets. It's always updated with regular files. I don't know if it's updated for block devices. So it likely won't give you any information for copying between /dev/zero and /dev/null, but it might work in your real use case.

Answer 2

You can also query the progress via the /proc interface.

# dd bs=1M if=/dev/mmcblk0 of=/dev/null &
# pidof dd
1358

So the information about this process is to be found in /proc/1358:

# ls -l /proc/1358/fd
total 0
lr-x------    1 root     root            64 Nov  2 09:16 0 -> /dev/mmcblk0
l-wx------    1 root     root            64 Nov  2 09:16 1 -> /dev/null
lrwx------    1 root     root            64 Nov  2 09:16 2 -> /dev/pts/0

Filehandle 0 is the if=/dev/mmcblk0, now where is the progress at?

# cat /proc/1358/fdinfo/0
pos:    2132803584
flags:  0400000
mnt_id: 17
# cat /proc/1358/fdinfo/0
pos:    2366636032
flags:  0400000
mnt_id: 17
# cat /proc/1358/fdinfo/0
pos:    2587885568
flags:  0400000
mnt_id: 17

In this fashion, for busybox dd, you can also derive progress from its fdinfo pos value.

That said, sending the USR1 signal in moderation should have minimal impact on performance. On an embedded system with few resources, polling the fdinfo might just as well have a similar impact.