[electroly]

I do something similar, but test out the performance before you commit to it. There is a massive chasm between how EFS is marketed and how it actually performs. EFS is the slowest possible way to store data in AWS, with painfully low per-client and aggregate throughput. We implemented EFS because it was easy and then immediately commenced the project to replace it with S3. The bare EFS solution didn't survive long enough to make it to the S3 project's finish; we had to add a layer of caching instances in EC2 to bring EFS throughput up to an acceptable level. We almost needed two layers of caching instances; i.e. EFS fans out to N caching instances, which then fan out to M caching instances, which then fan out to P workers, P >> M >> N. Because the EFS throughput is so poor, it could barely cope with the fan-out to the one layer of caching instances. Fortunately the S3 migration project finished before we got that far.

This announcement is, of course, the result of adding a caching layer on top of EFS. Naturally. But because they don't mention "throughput" and only mention "latency" I'm betting they have not used the cache layer to increase throughput.

I'll only ever recommend EFS if your data is very small and the throughput requirements are negligible.

[orf]

We had to move 300tb of EFS data into S3 in 40 hours, and this really showed how poor EFS performance can be.

The reason was metadata only, which caps out at about ~40mbs. Raw rear speed is great, but any metadata ops cap out quite hard.

We had to hack together a special NFS client to list the contents of the drive using as few metadata operations as possible, then have a separate step to copy the data.