2

I thought I could nuke all partitions of a drive by using dd if=/dev/zero of=/dev/sdX. In the past this has always worked for me, but in this case it is not working as expected.

#check the partitions
➜  ~ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   477G  0 disk 
├─sda1   8:1    0   512M  0 part /boot/efi
└─sda2   8:2    0 476.4G  0 part /
sdb      8:16   1  14.6G  0 disk 
├─sdb1   8:17   1   292M  0 part /media/james/Gentoo amd64 20190703T214502Z
└─sdb2   8:18   1   6.3M  0 part /media/james/GENTOOLIVE

#unmount and confirm the drive is still seen.

➜  ~ sudo umount "/media/james/Gentoo amd64 20190703T214502Z"
➜  ~ sudo umount "/media/james/GENTOOLIVE"
➜  ~ lsblk                                                   
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   477G  0 disk 
├─sda1   8:1    0   512M  0 part /boot/efi
└─sda2   8:2    0 476.4G  0 part /
sdb      8:16   1  14.6G  0 disk 
├─sdb1   8:17   1   292M  0 part 
└─sdb2   8:18   1   6.3M  0 part 

#Run dd

➜  ~ sudo dd if=/dev/zero of=/dev/sdb bs=3M                  
dd: error writing '/dev/sdb': No space left on device
2649+0 records in
2648+0 records out
8330620928 bytes (8.3 GB, 7.8 GiB) copied, 5.50879 s, 1.5 GB/s

#the partitions are still there!

➜  ~ lsblk

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   477G  0 disk 
├─sda1   8:1    0   512M  0 part /boot/efi
└─sda2   8:2    0 476.4G  0 part /
sdb      8:16   1  14.6G  0 disk 
├─sdb1   8:17   1   292M  0 part 
└─sdb2   8:18   1   6.3M  0 part 
➜  ~ lsblk

#after unplugging and replugging the drive, the old partition still mounts and still contains files. I was able to open several and read the contents.

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   477G  0 disk 
├─sda1   8:1    0   512M  0 part /boot/efi
└─sda2   8:2    0 476.4G  0 part /
sdb      8:16   1  14.6G  0 disk 
├─sdb1   8:17   1   292M  0 part 
└─sdb2   8:18   1   6.3M  0 part /media/james/GENTOOLIVE

What is really confusing me is that if I look in Gparted, the device is shown as 8GB unallocated, but this is a 16GB drive.

I ran badblocks -wsv, which passed but did so suspiciously quickly (minutes instead of hours). After unplugging and replugging, the drive shows up as /dev/sdc, and Gparted sees 14.56GB partition called "gentoo"

Testing with pattern 0xaa: set_o_direct: Invalid argument/0/0 errors)
done                                                 
Reading and comparing: done                                                 
Testing with pattern 0x55: done                                                 
Reading and comparing: done                                                 
Testing with pattern 0xff: done                                                 
Reading and comparing: done                                                 
Testing with pattern 0x00: done                                                 
Reading and comparing: done                                                 
Pass completed, 0 bad blocks found. (0/0/0 errors)
➜  ~ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   477G  0 disk 
├─sda1   8:1    0   512M  0 part /boot/efi
└─sda2   8:2    0 476.4G  0 part /
sdc      8:32   1  14.6G  0 disk 
├─sdc1   8:33   1   292M  0 part 
└─sdc2   8:34   1   6.3M  0 part

I'm guessing I should just put this flash drive out to pasture, but it seems to me such an odd sequence of events, I'm curious as to what sort of failure might have caused it (not really looking for a fix).

Edit: This was on Xubuntu 18.04

Edit2: After a reboot, zeroing works as expected. I guess it was just a temporary issue with the OS. I'm still curious about what sort of issue though.

Edit3: I spoke too soon, a reboot was not sufficient. I thought dd was working because it was taking a normal amount of time, but it seems not.

➜  ~ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   477G  0 disk 
├─sda1   8:1    0   512M  0 part /boot/efi
└─sda2   8:2    0 476.4G  0 part /
sdb      8:16   1  14.6G  0 disk 
├─sdb1   8:17   1   292M  0 part /media/james/Gentoo amd64 20190703T214502Z
└─sdb2   8:18   1   6.3M  0 part 
➜  ~ sudo dd if=/dev/zero of=/dev/sdb
[sudo] password for james: 
Sorry, try again.
[sudo] password for james: 
dd: writing to '/dev/sdb': No space left on device
30629377+0 records in
30629376+0 records out
15682240512 bytes (16 GB, 15 GiB) copied, 4232.1 s, 3.7 MB/s
➜  ~ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   477G  0 disk 
├─sda1   8:1    0   512M  0 part /boot/efi
└─sda2   8:2    0 476.4G  0 part /
sdb      8:16   1  14.6G  0 disk 
├─sdb1   8:17   1   292M  0 part /media/james/Gentoo amd64 20190703T214502Z
└─sdb2   8:18   1   6.3M  0 part

Edit 4: Ok, so dd did actually work, but lsblk did not update until I ejected and put back in.

➜  ~ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0   477G  0 disk 
├─sda1   8:1    0   512M  0 part /boot/efi
└─sda2   8:2    0 476.4G  0 part /
sdb      8:16   1  14.6G  0 disk

Edit 5: I checked dmesg and there is a warning about the disk not being properly mounted.

➜  ~ journalctl --dmesg --since="3 days ago" | grep sdb
Jul 09 19:59:27 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] 30595072 512-byte logical blocks: (15.7 GB/14.6 GiB)
Jul 09 19:59:27 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Write Protect is off
Jul 09 19:59:27 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Mode Sense: 43 00 00 00
Jul 09 19:59:27 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Jul 09 19:59:27 james-Latitude-E7470 kernel:  sdb: sdb1
Jul 09 19:59:27 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Attached SCSI removable disk
Jul 09 19:59:33 james-Latitude-E7470 kernel: FAT-fs (sdb1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
Jul 10 02:38:38 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] 30629376 512-byte logical blocks: (15.7 GB/14.6 GiB)
Jul 10 02:38:38 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Write Protect is off
Jul 10 02:38:38 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Mode Sense: 43 00 00 00
Jul 10 02:38:38 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Jul 10 02:38:38 james-Latitude-E7470 kernel:  sdb: sdb1 sdb2
Jul 10 02:38:38 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Attached SCSI removable disk
Jul 10 04:12:42 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] 30629376 512-byte logical blocks: (15.7 GB/14.6 GiB)
Jul 10 04:12:42 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Write Protect is off
Jul 10 04:12:42 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Mode Sense: 43 00 00 00
Jul 10 04:12:42 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
Jul 10 04:12:42 james-Latitude-E7470 kernel: sd 3:0:0:0: [sdb] Attached SCSI removable disk
Stonecraft
  • 799
  • 1
    The lsblk result doesn't show much change. Seems like no data was ever written to the disk at all. – 炸鱼薯条德里克 Jul 09 '19 at 00:47
  • Yes it does appear that way. But it said "no space left on device", which is what usually happens after I have successfully run dd. – Stonecraft Jul 09 '19 at 00:50
  • 1
    On a sidenote, if your goal was not to overwrite all data, you can also get rid of old filesystems partitions with wipefs like this https://unix.stackexchange.com/a/394999/30851 – frostschutz Jul 10 '19 at 08:40
  • did you only look at the first page of log output or ... scroll down? ;-) – frostschutz Jul 10 '19 at 08:48
  • As you point out in that answer, it is possible for other programs to grab data when the partition table is damaged, and this seems to have caused me issues in the past, hence my OCD zeroing. That is discussed here: https://unix.stackexchange.com/questions/487010/persistence-of-home-directory-after-deleting-all-partitions-and-reinstalling – Stonecraft Jul 10 '19 at 08:52
  • LOL good point, I will edit with something more useful – Stonecraft Jul 10 '19 at 08:53

1 Answers1

4

#the partitions are still there!

This part, at least, is still normal. You have to re-read partition table to update the partition information. You can trigger a re-read with

blockdev --rereadpt /dev/sdx

(alternatively sfdisk --re-read but that option has been removed from recent versions of sfdisk).

For everything else, yes it's possible for flash based storage to fail into a readonly mode. Strange things might also happen for other reasons, for example if the USB connection is not stable, the /dev/sdx device might be dropped, and redetected as /dev/sdy and all writes to /dev/sdx go to limbo, but I guess that would have showed in your lsblk.

Sometimes there are interesting error messages in dmesg, but all in all... if your USB stick failed you just have to get a new one, no way around it.

For sake of completeness, there is also this special case here (user error):

# dd bs=1M if=/dev/zero of=/dev/sdx
dd: error writing '/dev/sdx': No space left on device
7956+0 records in
7955+0 records out
8341966848 bytes (8.3 GB, 7.8 GiB) copied, 2.08568 s, 4.0 GB/s

So. Did this command write to a device?

Not at all. I don't even have a /dev/sdx. Instead it filled my /devtmpfs with 50% RAM sized regular file of zeroes. (I really should adjust my tmpfs size limits. If you do this on two instances of tmpfs, the system crashes because RAM is full.)

That happens when the device is missing entirely, or devicename mistyped, since dd does not check at all for existence beforehand, and if your machine has lots of RAM, and /dev not limited to a sane size like 10M, then you get this confusing result.

frostschutz
  • 48,978
  • 1
    I did not see your most recent edits when I started writing this answer. Hope it helps somehow anyway. Good luck. – frostschutz Jul 10 '19 at 08:17
  • See my latest edits. My best guess is that something went screwy with the original attempt at dd, which is why I could still read files from the disk. When I tried dding again after restart, all I had to do was eject and put back in for lsblk to update (which did not work the first time). So that seems pretty much in line with what you are saying. – Stonecraft Jul 10 '19 at 08:18
  • You can check your dmesg (or since you rebooted, logs) if there were any messages related to this device or USB in general. – frostschutz Jul 10 '19 at 08:19
  • Which logs would I check? I need to find a good linux logging guide that explains all the different standard logs and takes note of distro-specific differences vs conventions. – Stonecraft Jul 10 '19 at 08:23
  • 1
    @Stonecraft updated my answer with another idea I had, but it probably does not actually apply to you specifically... – frostschutz Jul 10 '19 at 08:26
  • @Stonecraft maybe journalctl --dmesg --since=yesterday or something of the like – frostschutz Jul 10 '19 at 08:35
  • I bet that is it (or at least part of the OS thought /dev/sdb was gone when it was not. And I did try this a couple times before my system froze and I had to restart, so that also fits with what you are saying. – Stonecraft Jul 10 '19 at 08:35
  • 1
    @Stonecraft It shouldn't be because your write speed was slow, and writing to devtmpfs should be blazing fast. (In my example 4GB/s). So it's still a mystery. – frostschutz Jul 10 '19 at 08:43