[jakjak123]

Have used SeaweedFS to store billions of thumbnails. The tooling is a bit clunky, but it mostly works. The performance is very good for small-ish objects (memory usage + latency), and latency remains consistently good into 99.9 percentiles. We had some issues with data loss and downtime, but that was mostly our own fault.

[hardwaresofton]

What issues did you run into? Not settling replication?

[jakjak123]

This was a couple of years ago now, but for example, some very minor amount of objects had not been replicated at all. This happened during heavy concurrent write traffic, and a couple of these race condition-ish bugs have been fixed over the years.