[petecooper]

One of my long-time favourite macOS applications -- iDefrag -- had support withdrawn shortly after APFS appeared. Reasons cited were lack of an APFS spec and increased System Integrity Protection.

I fear this is too little, too late to have iDefrag make a comeback. I understand defragmenting an SSD typically does more harm than good [edit: and I only defrag spinning drives), but nothing touched it for effectiveness on spinning drives.

https://coriolis-systems.com/iDefrag/

https://coriolis-systems.com/blog/2017/9/what-works-macos-10...

[r00fus]

I haven't defragged a disk in almost a decade. When is defragging an SSD ever a good thing?

I only have spinning rust in my (older) NAS or as backup drives.

[petecooper]

I don't defragment SSDs at all. I should've made that clear in my comment.

I have a multi-HDD system running to 6TB of storage, I can't run to the £ of an all-SSD system at the moment, but that's on the cards for the future, so it's spinning rust for me, for now.

[tinus_hn]

Probably often but there is no way to tell really because the controller has a mind of its own and they are all different.

[forapurpose]

> Probably often

Defragging an SSD is often a good thing? Why? It would seem to greatly increase wear for no benefit.

[akvadrako]

If your fragments are really small like 16kb then you could see a significant performance improvement, due to better predictive loading and packet overhead. This is clear from SSD benchmarks.

But I don’t see how this would ever realistically happen.

[forapurpose]

What about the massive effect on wear?

[tinus_hn]

You don’t know if there is wear. The controller is a separate computer. It may be clever and just remap blocks if you copy or move them around. It could be that defragmentation allows the system to work in a way that is better for the controller or allows that controller to place the blocks in a better way.

[tambourine_man]

HFS+ has had on-the-fly defrag for ages now. Did you have issues with it? What kind of files do you work with?

[sjwright]

Here's the essential business logic specification for HFS+ automatic de-fragmentation, for the curious.

It's actually quite a clever spec, because it takes advantage of existing efforts to read fragmented files to perform the majority of the de-fragmentation process.

I'm not sure if this spec applies to APFS or for SSDs. (With SSDs you're generally better off not defragmenting most of the time, because the performance penalty is far lower, but the write amplification has consequences.)

——————

When a file is opened on an HFS+ volume, the following conditions are tested:

  If the file is less than 20 MB in size
  If the file is not already busy
  If the file is not read-only
  If the file has more than eight extents
  If the system has been up for at least 3 mins

If all of the above conditions are satisfied, the file is relocated—it is defragmented on-the-fly.

http://osxbook.com/software/hfsdebug/fragmentation.html

——————

[petecooper]

A moderate amount of my work involves video training and the component parts to build, so raw video and audio. It's subjective, but there are times that video and audio stutters a bit and it's not down to the available CPU or RAM (plenty of headroom there).

[djrogers]

> I fear this is too little, too late to have iDefrag make a comeback

Defragging has been snake oil for more than a decade, anything that hastens it’s demise is a good thing.

[petecooper]

My computer (macOS Sierra, HFS (Journaled)) has 2x spinning drives of 3TB, and operations are noticeably quicker after an overnight defragmentation run, which I do twice a year or so. Anecdata, granted - but it works for my situation.

[pfranz]

If it works for you, then great, but here's an article from almost 10 years ago where Apple discourages defragging[1]. There's a bunch of other articles with more specifics showing that advanced defrag features from the 90s were built into the OS years ago[2].

[1] https://support.apple.com/en-us/HT1375 (last updated 2010) [2] http://osxbook.com/software/hfsdebug/fragmentation.html (from 2004)

[petecooper]

I just took a snapshot of one of my hard drives (2TB) from iDefrag:

https://imgur.com/bdGcXXC

The red slivers towards the lower middle are where the reported fragmented files are located, and the lower right is the fragmented files listed by number of fragments descending.

The anonymised .mkv files are training videos with multiple language and subtitle streams. They're exported to a scratch drive and then copied to the the drive they currently reside on.

After a full, offline defrag (including a b-tree rebuild) the legend for defragmentation is a neatly arranged list of that blue/grey colour, no red to be seen.

[sjwright]

Given that macOS automatically defragments sub-20MB files on the fly (which covers 99% of the files which affect perceived system performance) I'd wager money that your experience is 100% placebo.

I know from personal experience how powerful this placebo effect can be.

[petecooper]

>Given that macOS automatically defragments sub-20MB files on the fly (which covers 99% of the files which affect perceived system performance) I'd wager money that your experience is 100% placebo.

Perhaps. I can certainly tell the difference performance when an overnight defrag run has finished, and I have no beef with you to prove or disprove a point.

If you're ever in North Cornwall, UK: drop me a line and I'll show you a before and after over a mug of coffee/tea/etc.

[barrkel]

You have not used a file system exceeding 90% use with large files and lots of little files coming and going.

ZFS in particular completely falls off a cliff somewhere between 80% and 90%, due to the copy on write nature of ZFS always allocating and freeing small bits of space. That creates the little gaps all over the FS which murder performance when the big gaps run out.

[lathiat]

The old default for this was lower but most modern versions of the ZFS defaults don’t hit this until the vicinity of 96%

It’s not technically the little holes or even CoW that is the problem. It simply switched from a “first fit” to a “best fit” algorithm as it got full which was quite expensive to do the search for.