[orf]

At one point we had a ~560tb EFS disk that ran a variety of mixed workloads (large and small files). It was untenable - raw reading/writing IO is OK, but metadata IO hits a brick wall and destroys the performance of the whole disk for all connections (not just ones accessing a particular partition/tree/whatever).

In order to migrate off it and onto s3 I had to build a custom tool in rust that used libnfs directly to list the contents of the disk. We then launched a large number of lambdas to copy individual files to s3.

It was fun, but in my experience EFS is only good if you have a very homogenous workload and are able to carefully optimise metadata IO. I wouldn’t recommend it - s3 is just cheaper, faster and better.