[surrTurr]

> Hetzner volumes are, in my experience, too slow for a production database. While you may in the past have had a good experience running customer-facing databases on AWS EBS, with Hetzner's volumes we were seeing >50ms of IOWAIT with very low IOPS. See https://github.com/rook/rook/issues/14999 for benchmarks.

I set up rook ceph on a talos k8s cluster (with vm volumes) and experienced similar low performance; however, I always thought that was because of the 1Gi vSwitch (i.e. networking problem)?! The SSD volumes were quite fast.

[tehlike]

SSD volumes are physically on the same node, and afaik not redundant. The cloud vms are ceph clusters behind the scenes, and writes need to commit for 3+ machines. It's both network latency and inherent process latency

Additionally, hetzner has an IOPS limit of 5000 and write limit of some amount that does not scale with the size of database.

50G has the same limits as 5TB.

For this reason, people are sometimes using different table spaces in postgres for example.

Ceph puts another burden on top of already-ceph-based cloud volumes, btw, so don't do that.

[p_l]

RAID10 on local SSDs is pretty performant option, but yeah, it's per node.

[merpkz]

In my limited experience with rook-ceph it is strictly bare metal technology to deploy. On virtualization it will basically replicate your data to VM disks which usually are already replicated, so quite a bit of replication amplification will happen and tank your performance.