8

Related to this earlier question I asked, I'd like to know what filesystems will preferentially reuse blocks that are freed when files are deleted (if any exist)?

I am only interested in stable, well-established filesystems and would prefer those that can be the boot/root partition in a Debian system (though this is not essential).

edit:
Responding to a comment, I'd like this behaviour because the fs is for a VM and is contained in a sparse raw image file on the host. If new blocks are allocated, the image file gradually loses it's sparseness over time as files are created/deleted/modified, tending towards the 'non-sparse' size, even if the total storage used on the VM remains basically constant.

Community
  • 1
  • 2
    I dunno, but I can't help but think that "were these blocks used before they were freed?" is going to be much less of a priority for a filesystem to decide where to allocate than "do I have enough room at this point to store this file contiguously?" – Shadur-don't-feed-the-AI Nov 06 '12 at 10:07
  • ... Also, the ability to answer the question of "have these blocks been part of a file before?" might form a security risk. – Shadur-don't-feed-the-AI Nov 06 '12 at 10:08
  • I'd be happy enough if the fs cared more about contiguous files and security than reuse, as long as it searches for space always starting from the same end—therefore it would tend to reuse blocks. ext4 seems to do the opposite if I understand that page correctly. –  Nov 06 '12 at 11:18
  • 1
    @JackDouglas maybe you could motivate this with a reason why this would be desirable? In every case I can think of, it either doesn't matter or else the opposite behaviour would be preferable (e.g. on magnetic media use new blocks if possible to avoid bit rot, and on flash use new block if possible for wear levelling). – Celada Nov 06 '12 at 14:55
  • Is there any reason you are making the VM disk much bigger than you actually need / want? Why not start with a small disk and just add disks on demand? Adding virtual disks to a VM is quite trivial. Alternatively you could periodically do shrink-maintenance, e.g. run something like zerofree (write 0 to all unused blocks) inside the VM and an external tool to shrink the virtual disk. – jw013 Nov 06 '12 at 15:14
  • @jw013 because I have dozens of them spread over many hosts. I agree that is an option, but the maintenance is something I'd like to avoid if possible. –  Nov 06 '12 at 15:18
  • 1
    @JackDouglas thanks for the motivation. Good one, I didn't think of that. Hopefully someone has an answer. The only suggestion I have is, use a SAN instead of block devices (NFS root on the VM)? – Celada Nov 06 '12 at 15:21
  • @Celada you mean NFS on the host? Interesting suggestion, thanks :) –  Nov 06 '12 at 15:27
  • 1
    I mean, one VM (or perhaps a dedicated physical machine) that has a huge non-sparse disk and acts as a file server for all the other VMs. If you want high availability then make the file server a clustered system using DRBD or iSCSI for the storage and then you're really cooking :-) Do beware if you've never done clusters before that redundancy always adds complexity. – Celada Nov 06 '12 at 15:57

2 Answers2

3

If you are looking for stable and well established, then I think you are out of luck. Otherwise, btrfs behaves this way.

You also might want to look into whether your virtual disk supports TRIM to free the blocks and make sure you enable discard in your fs.

You also might consider just allocating smaller disks initially more sized to your anticipated needs; most filesystems can be easily expanded online these days should you need to increase it in the future.

psusi
  • 17,303
  • 1
    Thanks. "You also might want to look into whether your virtual disk supports TRIM": that was the subject of the linked question btw. –  Nov 06 '12 at 17:47
2

xfs is stable and well established and has the characteristics I'm looking for — it reuses the 'just freed' blocks very quickly:

xfs, ext2, and ext3 behaved as I expected. They ended up using 316 MiB, 701 MiB, and 1351 MiB, respectively. btrfs used up to 2763 MiB, presumably because it's a copy-on-write filesystem. It didn't allocate this space at once, but allocated some blocks in a linear manner and then stalled for a while. ext4 was a surprise. It grew almost linearly and capped at 19 GiB, or 93% of the disk's size. While this may be well suited for solid state disks without TRIM support, it is not for rotating disks which show a higher latency and lower throughput on the outermost regions.

enter image description here