Hacker News new | ask | show | jobs
by ww520 4331 days ago
CopyCat looks really good for HA and fault-tolerance. Not so sure about its scalability since all writes go through the single leader node. It's more appropriate for the use of maintaining metadata of a distributed system, rather than maintaining the data themselves.

For your requirement, can you use S3 or something similar?

2 comments

Thanks! The software I am writing needs to be (easily) installed at the customer site. So, S3 won't work for me. Are there any open source S3 clones that are easy to setup?
If you mean that you need something that supports the S3 API, you could try S3Proxy[0]. It basically presents the S3 API over a variety of different storage backends.

[0] https://github.com/andrewgaul/s3proxy

Riak CS is S3 compatible.

I have a very similar problem and end up writing a python script that copies the files to cassandra and Nginx + Lua to serve them using lua-resty-cassandra. The read/write throughput is still not as good as I was expecting though.

You could try OpenStack's Swift project.
That is a "design feature" of raft, which allows it to lack the unbelievable complexity of Paxos, Multi-Paxos, or something like ZAB (yet another paxos variant).