Ipsa scientia potestas est

A few weeks ago someone over at forums.mactalk.com.au posted a thread on the forum regarding the necessity and/or validity of defragmenting his hard-drive. He didn’t cite Apple’s official stance on the fragmentation issue and he more importantly didn’t give us any insight into the kind of average disk utilization the target hardware underwent before defragmenting. He also didn’t show us the level of fragmentation his target disk had pre and post-defrag.

In this post I will attempt to provide a bit more insight into the necessity of defragmenting, as well as provide a bit more detail as to the state of the drive pre and post-defrag.

Environment:

• Macbook Pro 15″ - 1.83 GHz Intel Core Duo - 512 MB 667 MHz RAM
• 80Gb HD
• Disk usage heavy, daily - Large files (Bit Torrent going constantly)
• Disk capacity hit/approached regularly, large files deleted often.

diskutil output for /:

herecy:~ dmess0r$ diskutil info /
   Device Node:        /dev/disk0s2
   Device Identifier:  disk0s2
   Mount Point:        /
   Volume Name:        Macintosh HD

   File System:        Journaled HFS+
                       Journal size 8192 k at offset 0xa701000
   Owners:             Enabled
   Partition Type:     Apple_HFS
   Bootable:           Is bootable
   Media Type:         Generic
   Protocol:           SATA
   SMART Status:       Verified
   UUID:               546B569B-63F1-3005-9D0D-276370A731A8

   Total Size:         74.2 GB
   Free Space:         9.7 GB

   Read Only:          No
   Ejectable:          No

Now that we have set the stage, it is important to run some disk and filesystem benchmarks to get a baseline. Without the baseline we really won’t be able to compare much.
In the mactalk post, the individual performed an unreliable performance benchmark by simply timing the starting up of applications. Filesystem performance must be evaluated in a controlled environment and not be subject to personal habits or bias.

We need to do our best to isolate the FS, run a controlled benchmark, defragment, and then run the controlled benchmark again. If we can prove through benchmarking that there is some noticible FS performance gains, then defragmentation may very well be worth it. Lets find out, shall we?

I picked up a copy of iDefrag, and checked out my system, heres a screenshot of how my system looked pre-defrag:

I have put the key, or legend as I call it, next to the main screen so you can get an idea of how the files are allocated on the4 filesystem. Next I have the statistics screen:

Notice that I only have 877 fragmented files on my filesystem. Given my disk usage combined with the rather large files I work with regularly, to only have 877 files fragmented on the entire disk is testament to HFS+’s aggressive stance towards anti-fragmentation of files. Even so, I decided to see if I could squeeze any performance out of the filesystem by defragmenting the drive.

Next, I IOZone benchmarked the filesystem several times and came up with a statistically sound set of results. Bear in mind that I did not use any direct disk calls MS_SYNC, or O_SYNC. particularly O_SYNC (the “-o” option) as the point of this test was to test the filesystem in its native environment, not necessarily to test the disk itself.

Here were the results:

• Pre Defrag - Perf - Read
• Pre Defrag - Perf - ReRead
• Pre Defrag - Perf - Backward Read
• Pre Defrag - Perf - Random Read
• Pre Defrag - Perf - Fread
• Pre Defrag - Perf - ReFread
• Pre Defrag - Perf - Stride Read
• Pre Defrag - Perf - Write
• Pre Defrag - Perf - ReWrite
• Pre Defrag - Perf - Random Write
• Pre Defrag - Perf - Record ReWrite
• Pre Defrag - Perf - Fwrite
• Pre Defrag - Perf - ReFwrite

Now that I have a reasonable baseline as to disk performance, and a good snapshot of what my fragmentation looks like, it is now time to defrag the drive and see if we can squeeze out any additional performance.

I picked up a copy of Coriolis CDMaker v2.0.0 and also added a copy of iDefrag. I booted up into the limited environment and proceeded info iDefrag.

It is important to note that I selected the Algorithm entitled “Full Defrag”, (splat five) as well as under the iDefrag Preferences, under the “Compact” tab, I selected to “Compact B-Tree files” and “Rebuild rather than just compacting*”. Under the “Metadata” tab, I also selected to “Compact B-Tree files” as well as “Rebuild rather than just compacting”.

Once these items were all set to go, I selected the appropriate volume and let it fly.

I left home, and arrived back approximately 12 hours later to find that it had successfully defragmented my drive. Here is the screenshot of the main iDefrag window post-defragment.

The result looks much cleaner to me, but what will this defragmentation mean to overall filesystem performance?

I decided to re-run the IOZone benchmark with the same options which were used on the first go-round, take a sample of one of the rounds of output, and here is that output:

• Post Defrag - Perf - Read
• Post Defrag - Perf - ReRead
• Post Defrag - Perf - Backward Read
• Post Defrag - Perf - Random Read
• Post Defrag - Perf - Fread
• Post Defrag - Perf - ReFread
• Post Defrag - Perf - Stride Read
• Post Defrag - Perf - Write
• Post Defrag - Perf - ReWrite
• Post Defrag - Perf - Random Write
• Post Defrag - Perf - Record ReWrite
• Post Defrag - Perf - Fwrite
• Post Defrag - Perf - ReFwrite

Does this new data reveal any noticeable filesystem performance gain? The answer which you all have been waiting for is.

Question: Does defragmenting a harddrive which undergoes rigorous largefile, Bit Torrent activity gain any speed through defragmentation?

Answer: No. There is no noticeable filesystem performance gain.

On a separate note, which I will be elaborating on later, is that there may be some performance gains by aligning the logical blocks of the filesystem to the physical blocks of the disk. This way logical blocks do not cross physical boundaries. I have used this technique on enterprise class servers, SAN and NAS systems with significant results.

Notice the bulge in the center of the graph like a spine? This is showing us that using a record size of ~256Kb or ~512Kb results in significant throughput. But why? I will dig in deeper later, but this is a teaser pointing out that 256 or 512 may be a multiplier in our correlation between physical boundaries and logical ones. More later.