Or, run Riak with their S3 compatibility layer. Riak is extremely stable and the work Basho has done to make a truly robust distributed database is significant.
Also ceph (& swift) are known to scale well in prod. clusters with over 30+ PB of data (at least looking at CERN's cluster) and the latest version of RGW does support geographic redundancy for S3 like apis
+1 for Ceph. We're running several ~3.5 PB clusters in production. We've not taken advantage of the new RGW features in Jewel, but it works well as an object storage solution.
+1 on RiakCS. They now call it RiakS2 for kicks. The scalability and reliability of their server is insane. You just can't beat Erlang software in that regard.
Unfortunately, Basho has been so successful with their TSDB and KV products that they have basically put S2 on maintenance mode. They are still "supporting" it, but no new features. I was hoping this Minio tool could do something similar, but with a single daemon is a single point of failure. Unacceptable for serious deployments.
There's also Skylable's SX Cluster if you use the libres3 daemon with it. Been using it for over a year with no problems. Set, forget, add more nodes when I need more disk.
Everyone's got their s3 of choice, always good to have more options on the table.
Ran Riak CS in production and had constant issues. It's not terrible but it's also not ideal. I would caution against anyone depending on it for mission critical systems. Many of the failure modes are undocumented.
Minio is deliberately designed this way. Cloud native applications require strict multitenancy. Minio's approach is to build just enough to meet a single tenant's requirement. Deploy one minio server per tenant or user or customer .. whichever fits you the best. This will allow you to upgrade, customize or bug fix in isolation.
To replicate for HA, use "mc mirror -watch SOURCE TARGET" command to pair them up. If you have multiple drives (JBOD), you can eliminate RAID or ZFS and use Minio's erasure code to pool them up. Distributed version is also in testing at the moment. It should be out in a month.
ceph - http://docs.ceph.com/docs/master/radosgw/s3/
swift - https://wiki.openstack.org/wiki/Swift/APIFeatureComparison#A...