[Rezo]

I wonder if Amazon's new managed EFS service would make sense? It's exposed as a NFS mount to the OS. The claims are:

- Up to thousands of Amazon EC2 instances, from multiple AZs, can connect concurrently to a file system.

- Data is stored redundantly across multiple AZs.

- Low, consistent latency.

- Multiple GBs per second.

[sytse]

EFS is pretty great. But for our use case there are a few drawbacks:

1. It doesn't scale to the size we need for all repositories, so we still need sharding.

2. It is expensive, $300/TB-month

3. You're still sending all traffic over the network, we prefer the latency of local storage we can achieve by developing Gitaly https://gitlab.com/gitlab-org/gitaly

[jamesmiller5]

Why develop Gitaly instead of using consensus (etcd, zookeeper, etc.) for writes to fast storage like ssds and the normal git binary? As far as I can tell it's adding a lot of complexity by caching high level RPC calls but still doesn't really address multi-master read after write consistency or coordination.