4

I am using the dd tool to make back-ups of Raspberry Pi SD card images to my local machine. When cloning one of these cards, the system drive ran out of free space. My machine had 112 GB of free space available, and the SD card was 128GB. The command I issued to copy the card is as follows:

sudo dd if=/dev/mmcblk0 of=pi_backup.img status=progress

dd did not produce any sort of error or warning when the system ran out of space, and reports that the entire image was copied to my local machine. The size on disk is just over 128GB. The system storage is totally full.

dd output:

128126352384 bytes (128 GB, 119 GiB) copied, 1524 s, 84.1 MB/s
250347520+0 records in
250347520+0 records out
128177930240 bytes (128 GB, 119 GiB) copied, 1524.57 s, 84.1 MB/s

df -h output before copy (truncated /dev/loop devices for brevity):

Filesystem      Size  Used Avail Use% Mounted on
udev            7.7G     0  7.7G   0% /dev
tmpfs           1.6G  3.0M  1.6G   1% /run
/dev/nvme0n1p7  202G   79G  112G  42% /
tmpfs           7.7G   34M  7.7G   1% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           7.7G     0  7.7G   0% /sys/fs/cgroup
/dev/nvme0n1p1  186M   94M   93M  51% /boot/efi
tmpfs           1.6G   20K  1.6G   1% /run/user/125
tmpfs           1.6G  132K  1.6G   1% /run/user/1000
/dev/mmcblk0p1  253M   50M  203M  20% /media/tdarby/boot
/dev/mmcblk0p2  118G   15G   99G  13% /media/tdarby/rootfs

df -h output after copy:

Filesystem      Size  Used Avail Use% Mounted on
udev            7.7G     0  7.7G   0% /dev
tmpfs           1.6G  3.0M  1.6G   1% /run
/dev/nvme0n1p7  202G  199G     0 100% /
tmpfs           7.7G   34M  7.7G   1% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           7.7G     0  7.7G   0% /sys/fs/cgroup
/dev/nvme0n1p1  186M   94M   93M  51% /boot/efi
tmpfs           1.6G   20K  1.6G   1% /run/user/125
tmpfs           1.6G  136K  1.6G   1% /run/user/1000

ls -lh pi_backup.img after copy:

-rw-r--r-- 1 root root 120G Jan 12 15:28 pi_backup.img

Attempting to write to disk after the copy:

~$ echo "hello" >> out.txt
bash: echo: write error: No space left on device

However, dd never errored. In fact, if I run dd again afterwards, it keeps copying a new image as if the disk is not full!

As shown from the logs, there was originally 112G available on the disk, and the size of the file after copy was 120G.

It's my understanding that dd should error out when its output disk becomes full, giving something along the lines of dd: error writing ‘/dev/sdx’: No space left on device. No such warning was produced.

I am concerned that dd has overwritten adjacent files or partitions on my disk. Even now, with the disk still full, I can run the dd command again and it makes no complaints of the disk being full, it just keeps copying.

What is the expected behavior here?

Tyler Darby
  • 41
  • 2
    Please paste df -h result as and ls -lh pi_backup.img so we can verify what your saying is true. – spinkus Jan 12 '23 at 19:56
  • @spinkus I have added the requested command outputs. Thanks! – Tyler Darby Jan 12 '23 at 20:34
  • try sudo sh -c 'echo "hello" >> out.txt' – ilkkachu Jan 12 '23 at 20:37
  • @ilkkachu This command ran without error and the file was successfully written to disk (or at least I can cat it afterwards). – Tyler Darby Jan 12 '23 at 20:38
  • "In fact, if I run dd again afterwards, it keeps copying a new image as if the disk is not full!" -- "with the disk still full, I can run the dd command again and it makes no complaints of the disk being full, it just keeps copying." -- do you mean running the copy to the same file or to another one? How far does it get to? – ilkkachu Jan 12 '23 at 20:38
  • @ilkkachu I changed the output file to another file (of=pi_backup2.img) and it copied several hundred megabytes before I killed it out of fear that I was overwriting other things on my system. – Tyler Darby Jan 12 '23 at 20:41
  • dd isn't magic, if you tell it to write to a file, it'll write to a file through the filesystem, same as any other command. (If the filesystem would let file accesses trash other files, then nothing would save you.) – ilkkachu Jan 12 '23 at 20:44
  • 2
    Anyway, see Reserved space for root on a filesystem - why? – ilkkachu Jan 12 '23 at 20:44

1 Answers1

1

Careful, a gigabyte (GB) is not the same as a gibibyte (GiB, G).

  • 1 gibibyte == 1 GiB == 10243 bytes == 230 bytes == 1,073,741,824 bytes
  • 1 gigabyte == 1 GB == 10003 bytes == 109 bytes == 1,000,000,000 bytes

df suffixes are using the power of 2 variants, see how dd was kind enough to tell you both (128 GB, 119 GiB).

Your filesystem had 202G - 79G =~ 123 GiB free, a few of them reserved for root which explains why the Avail column showed 112G(iB), so that's enough to hold the 119GiB of your device.

So dd did succeed, and you see there is still 202G - 199G =~ 3GiB free, even though those are only available to root processes (hence the 0 in the Avail column).

Stéphane Chazelas
  • 544,893