2

I just read this article about the virtually non-existent disk fragmentation on *nix filesystems.

It was mentioned that due to the way ext handles writing data to the disks, fragmentation may only begin manifesting on hard drives that are at least 80%, where the free space between the files starts to run out.

On how to deal with this fragmentation, the final paragraph reads:

If you actually need to defragment a file system, the simplest way is probably the most reliable: Copy all the files off the partition, erase the files from the partition, then copy the files back onto the partition. The file system will intelligently allocate the files as you copy them back onto the disk.

That sounds illogical to me. Because as far as I understand, when copying all files back to the erased drive, a similar process should take place where files are written and written with gradually decreasing portions of free space between them, to the point where fragmentation will manifest again.

Am I right on this one?

jasonwryan
  • 73,126
pilau
  • 123
  • 2

1 Answers1

3

What you have read is true. File systems become fragmented over time - as you write more of your epic screenplay, or add to your music collection, or upload more photos, etc, so free space runs low and the system has to split files up to fit on the disk. In the process described in the excerpt you posted, the final stage, copying the files back onto the recently cleaned disk, is done sequentially - so files are written to the file system, one after another, allowing the system to allocate disk space in a manner that avoids the conditions that led to fragmentation in the first place.

On some UNIX file systems, fragmentation is actually a good thing - it helps to save space, by allocating data from two files to a single disk block, rather than using up two blocks that would each be less than half filled with the data.

UNIX file systems don't start to suffer from fragmentation until nearly full, when the system no longer has sufficient free space to use as it attempts to shuffle files around to keep them occupying contiguous blocks. Similarly, the Windows defragmenter needs around 15% of the disk to be unused to be able to effectively perform its duty.

D_Bye
  • 13,977
  • 3
  • 44
  • 31
  • Fantastic answer. Files are copied sequentially, so no fragmentation occurs. Pretty neat :) – pilau Sep 27 '12 at 09:18
  • Another note is that in filesystems such as ext2/3/4, some portion of the disk (5% by default) is reserved. One use of this reserved space is to combat fragmentation. – jordanm Sep 27 '12 at 15:34
  • @jordanm - a good point. The same is true of UFS and UFS2 (where the reserved space is 8% by default on FreeBSD). It's probably true on other file system types, too. – D_Bye Sep 27 '12 at 15:55
  • In what way is that free space used to fight fragmentation? – pilau Sep 27 '12 at 18:15
  • @pilau see this answer: http://unix.stackexchange.com/a/7965/13796 – jordanm Sep 27 '12 at 18:50
  • @pilau - ever thought about tackling the mess in the garage, but don't know where to start because there's just so much stuff? I mean, there's barely room to open the door, let alone to do any meaningful sorting... The file system needs a bit of free space, so it can shuffle data blocks around and thus keep files occupying contiguous blocks. This becomes increasingly difficult the less free space is available. – D_Bye Sep 28 '12 at 07:19