[devplusops]

I tried minio and was really impressed with how easy it is to get started.

But I couldn't figure out how to scale it or make it reliable. Having my entire system depend on a single server not going down is a non-starter for me. Do you consider this use case in-scope?

I've been looking at Skylable instead. While setting it up is a little clunky, it's the only open source package I've found that explicitly supports replication across multiple datacenters - which is even more interesting to me than simply across servers within a data center.

[kparthas]

For reliability, you could start another minio server process on a different machine and use "mc mirror" subcommand (https://docs.minio.io/docs/minio-client-complete-guide#mirro...) to asynchronously copy out data. In this release, one needs to run "mc mirror" periodically using cron(8) or an equivalent tool in their platform. We are looking at providing "-c" option to "mc mirror" subcommand, which performs continuous replication. i.e, there would be no need run it periodically.

Minio encourages micro-storage architecture and scalability is achieved by deploying many minio server instances.

Hope that helps.

Disclaimer: I work at Minio.

[kparthas]

To clarify my previous reply, in a cloud environment, scalability and multi-tenancy is achieved by spawning many instances of minio server, one per tenant. We are also working on a distributed minio server at the moment.

[devplusops]

Cool!

The continuous async replication feature you mention would be ideal, especially if it's setup so that I can use consul/etcd/... to automatically elect a "master". If you also added (1) a feature so that writes can be redirected to another instance, and (2) a feature so that I can direct SOME of my reads to a master (reads where I care about consistency), then you'd have a not too difficulty to implement - but also very powerful - distributed version of minio. Maybe something along those lines is what you are planning.

Many instances of minio makes a lot of sense to me, and it seems light weight enough (i.e. resources like memory) that it would work. The only deal-breaker feature which looked like it was missing was the ability to set a disk quota. In multi-tenant environments it's important to have resource limits for reliability reasons. I considered putting each minio instance in it's own ZFS volume to handle disk quotas, but I would have had to write a lot of glue code to make that work from a deployment perspective.

Great work on minio - it's the least painful to use S3 server implementation I've run across so far.

[unlocksmith]

Good point @devplusops. Disk quota should be built into Minio. Will add it to the roadmap.