[nickcw]

This is the problem IMHO

> We also looked up whether our HBA used TRIM in its current configuration. It turns out, in RAID mode, the HBA did not support TRIM. We did do some trim-enabled testing with a different machine, but these results are hard to compare fairly. In any case, we can't currently enable TRIM on our production systems.

In our experience SSD write performance goes to sh*t if you don't regularly TRIM them.

Running fstrim once a day is enough to keep them healthy.

RAID cards not passing TRIM is a big problem for us too...

(Experience from day job at Hosting Provider)

[masklinn]

> In our experience SSD write performance goes to sht if you don't regularly TRIM them.

Interesting, is that because of the load? It seemed "modern" SSDs have GCs good enough that trim isn't quite necessary anymore to ensure good performances in consumer loads.

> RAID cards not passing TRIM is a big problem for us too...

Are there NVMe RAID cards? I assume they'd necessarily pass the command along considering deallocate* is just one parameter/option of the DATA SET MANAGEMENT command, or do RAID cards just drop the entire command?

[takeda]

> Interesting, is that because of the load? It seemed "modern" SSDs have GCs good enough that trim isn't quite necessary anymore to ensure good performances in consumer loads.

A drive has no way to tell whether filesystem is using a given block or not. TRIM is a way for the filesystem to tell it that. So I would imagine the GC that you're referring to is working on the blocks marked with TRIM.

BTW, besides running fstrim from cron on Linux, you can also use discard flag to mount the drive, so the filesystem sends TRIM command when files are deleted.

[wtallis]

> So I would imagine the GC that you're referring to is working on the blocks marked with TRIM.

Not necessarily. Since flash doesn't support in-place modification of data, any change to a portion of a file (or other FS data structure) that writes less than a contiguous 16MB (depending on the flash) will create a need for GC on the drive with or without TRIM. You can put a drive into a state of needing to do a lot of GC even without changing the quantity of live data.

[wtallis]

Modern consumer SSDs still benefit from TRIM, but are mostly able to keep up with GC just fine without it when subjected to typical consumer IO workloads, which are full of idle time for the drive to catch up. But if you fill an SSD to the brim, it'll slow down, and the cheapest SSDs will slow down a lot.

There are no hardware RAID solutions for NVMe, though there are now several hardware platforms supporting software RAID for NVMe devices in their motherboard firmware so you can boot from a NVMe RAID array. As with any other RAID solution, translating trim/unmap/deallocate commands takes a bit of effort, and less mature NVMe RAID solutions don't necessarily bother.