Why does baobab estimate my usage as less than half compared to du or pan_quota?

Question

On a panfs filesystem:

pan_quota says I'm using 8.67 GB.
du -hcsx says I'm using 8.1 GB.
baobab says I'm using 3.3 GB.

This is for my home directory, with a 8 GB soft and 10 GB hard quota. What might cause the modest discrepancy between duand pan_quota? And what may cause the huge discrepancy between baobab and either du or pan_quota?

The difference is not due to one or two specific files, but propagates down and is present for smaller and larger directories alike.

Do you have a lot of tiny files? Could be a logical v. physical size difference; what does du --apparent-size -hcsx give? — derobert, Apr 22 '16 at 17:34
Have you read Why are there so many different ways to measure disk usage?? — Gilles 'SO- stop being evil', Apr 22 '16 at 22:17
@derobert 34301 files, blocksize 64 kB. The du --apparent-size -hcsx command you suggest gives the same answer as baobab. — gerrit, Apr 22 '16 at 22:39
@derobert I have partly. I thought the differences I found are too large to be explained by it, but perhaps I was wrong. Block size is apparently 64k. — gerrit, Apr 22 '16 at 22:41
@gerrit That's what's going on then, it's small files eating an entire block. A 64k block size is pretty big. The question Gilles linked explains in depth. — derobert, Apr 22 '16 at 22:41

score 1 · Accepted Answer · edited Apr 13 '17 at 12:36

The disk space used by a file includes more than the size of the contents. Most filesystems use fixed-size blocks; if a file's size isn't a multiple of the block size, then the last block is only partially filled. du counts the total number of blocks, which is usually a little more than the sum of the file sizes since it also includes the part of the last block that isn't filled.

I think the difference between pan_quota and du -h is due to the fact that the PANFS filesystem doesn't report its block size accurately. The JASMIN FAQ suggests using pand_du instead of du.

It seems that on your machine, Baobab is displaying the sum of the file sizes. That's the number you get with du --appparent-size. Baobab displays the actual disk usage on my machine (but with 1GB = 10⁹ bytes, whereas du -h uses 1GB = 2³⁰ bytes); I think this is because you're using a version older than this change

commit dbcbe27e0452eeacbc4a253f1b0b1a06708834de
Author: Paolo Borelli <pborelli@gnome.org>
Date:   2012-01-07

    Use the STANDARD_ALLOCATED_SIZE file attribute.

but I haven't traced it down to make sure.

For more information about disk usage discrepancies, see Why are there so many different ways to measure disk usage?

I neglected to mention that I don't run baobab on the host machine (where it is unavailable), but through sshfs, so perhaps that prevents baobab from performing an accurate calculation. — gerrit, Apr 23 '16 at 20:33

Why does baobab estimate my usage as less than half compared to du or pan_quota?

1 Answers1