1

I want to look at a raw block device (as in /dev/sda) and know how much space is being used on the drive.

I don't know what type of file system is on the harddrive, nor do I have any way of figuring this out. A harddrive can obviously not be "empty", but what I do know is "empty space" is represented as bunch of zeroes (as in, output from /dev/zero, not the ASCII character 0).

How do I scan a block device, and then get an output telling me how many of the blocks are only filled with zero values?

IQAndreas
  • 10,345
  • This would take a really really really long time, almost as long as copying the entire drive if it were completely full. Each bit on the drive would have to be checked. You really should find out what filesystem it is; with that data in hand, all that needs to be checked is the superblock, quickly, and you don't have to do it manually. – Wildcard Oct 14 '15 at 06:19
  • Did you try sudo blkid? That should tell you the filesystem type. (Or a filesystem type for each partition, if there is more than one.) – Wildcard Oct 14 '15 at 06:20
  • 1
    @Wildcard I am very ready to check the drive bit by bit (in fact, that's what I was expecting to do, no way around it). – IQAndreas Oct 14 '15 at 06:22
  • Does testdisk or photorec do what you want? (http://www.cgsecurity.org/). If you are running debian, they are in the testdisk package. Probably packaged for other distros too. I presume you ultimately have some other task (like recovering files) beyond just getting the disk utilisation. – cas Oct 14 '15 at 06:25
  • 1
    @Wildcard The question as I wrote it gets to the core of what I need to learn to continue; what I am trying to do is more complicated. Currently, the drive is using ZFS, but it used to be a different file system that was overwritten. I am trying to recover data from the old file system. In the process, I want to store an image of the block device, so I can recover it on a separate machine at a later time, but I don't want to store the blocks that contain no data. I do not want to compress the image. – IQAndreas Oct 14 '15 at 06:27
  • 1
    what you are asking for (don't store blocks that contain no data) is file-system dependent. NULs are perfectly valid data in files and without the filesystem info there is no way to distinguish between blocks that just happen to be full of NULs and blocks that belong to a file that contains lots of NULs. Similarly, there's no way to distinguish between unused blocks that contain non-NUL data (e.g. that used to belong to a deleted file) and actual files. – cas Oct 14 '15 at 06:32
  • partclone (http://partclone.org) may do what you want...but will probably get horribly confused by the fact that the partition table has been changed because the disk is now being used by ZFS. – cas Oct 14 '15 at 06:36
  • I appreciate the help, but stop trying to solve my exact problem, guys. I'm trying to learn a new skill (looping through a device block by block) that I can re-use outside of the exact situation I faced with right now. Just answer the question asked. – IQAndreas Oct 14 '15 at 06:39
  • the thing that you don't want to do (compression) IS the least hassle solution. anything else you will have to hack up yourself (e.g. read in a block at a time, discard the block if it's only NUL bytes) because there are no existing tools to do what you want (and, as i said, that's essentially impossible because it's file-system dependant). – cas Oct 14 '15 at 06:43
  • I like how nobody's actually trying to answer your actual question and is just being pedantic. https://unix.stackexchange.com/q/458293/19012 is a similar question with actual answers – endolith Jan 21 '20 at 05:37

1 Answers1

0

what I do know is "empty space" is represented as bunch of zeroes

This is wrong. While many disks do start out as all-zeros, they don't stay this way. Most operating systems do not wipe a disk sector to all-zeros if it stops being in use. They just leave it containing whatever it last contained until the sector is used again.

SSD devices are likely to have more all-zero sectors, because TRIM results in an all-zero block, but that's not a guarantee that unused sectors will be zeroed out by any means, just the ones that have been part of a larger unused block for long enough.

The only way to detect unused space on a filesystem is to analyze the filesystem structure. The unused space is whatever isn't used by the filesystem.

Gilles 'SO- stop being evil'
  • 829,060