Hacker News new | ask | show | jobs
by huntaub 584 days ago
Thank you! That’s exactly my hope! How can we make these technologies as easy as possible for teams to use so that they can build the best <training> <analysis> or whatever applications without becoming storage experts.
1 comments

How does it compare perf-wise to FSX Lustre backed by S3? Other than pay-as-you-go are there any functional differences?
Ultimately, we should be able to match the performance of FSx for Lustre. With a scale-out protocol (mirroring the Lustre protocol), it's relatively simple to achieve scaleable throughput and IOPS out to the level of 100s of GiB/s and millions of IOPS (we aren't doing this today, but I anticipate being here within a month or two).

Functionally, we're not running DRTs like Lustre does in order to synchronize from S3. We will support the ability to pre-load the cache (like a DRT), for customers who can't handle the lazy-loading latency, but unlike Lustre, we can't run out of capacity in the hot tier, and we will automatically delete data from the hot tier when it's not in use. I know that DRT management can be complex for customers.