[viraptor]

> It turns out that really unstable hardware/networks like to expose bugs.

This sounds like a weird complaint to be honest. If you verify that your hardware is unstable, how can you expect the software not to fail and corrupt data?

> Also, read the Riak documentation and you'll find the corruption recovery documentation among other hints at common failures and limitations.

I'm not sure how that's a negative thing. You think about and document recovery even if you don't expect things to fail.

[nicolast]

> If you verify that your hardware is unstable, how can you expect the software not to fail and corrupt data?

No such thing as 'stable hardware' at scale.

[viraptor]

Ok, but then there's also no such thing as no corruption at scale.

If you accept imperfect hardware, you will get errors written to the drive. A single node of a database will get corruption by definition in that case. We're taking about RocksDB specifically here, so it is only one processing node.

How did you expect it to behave instead?

[StreamBright]

There is, just the error rate is different between a SAN backed openstack cluster vs AWS for example. EC2 is reliable compare to what hw just described above.