Hacker News new | ask | show | jobs
by tjoff 939 days ago
> However, the new storage class does open up an exciting new opportunity for all modern data infrastructure: the ability to tune an individual workload for low latency and higher cost or higher latency and lower cost with the exact same architecture and code.

I get it, but at the same time that is also what you lost when you locked yourself in with a particular vendor.

4 comments

There's not much to the S3 API, and data import/export even at massive scale is available with Snowball. Sure, there's many other AWS services that aren't available at other vendors, but blob storage is commodified at this point.
Exporting data from S3 is ludicrously expensive, even with Snowball it’s $30/TB just for network egress.
If I can get all of my data out of S3 for $30, then whatever the cost is for an enterprise with 1000TB, it must be within reason.
> I get it, but at the same time that is also what you lost when you locked yourself in with a particular vendor.

What are other viable practical alternative solution(s)?

Storage adapter to talk S3 compatible to target, assuming you're not relying on vendor specific extensions or behavior (ie this).

Off the top of my head, Backblaze B2, Cloudflare R2, etc are S3 compatible, and Minio locally.

https://www.google.com/search?q=s3+compatible

There are no vendor specific extensions or behavior here, are there? Isn't it just a different billing structure?
I suppose “super low latency” is behaviour, in the sense that “a large enough quantitative difference is a qualitative difference”. If you rely on the perf and only S3 provides that, then you effectively are locked into S3 implementation
Notifications, for event processing architectures aren't part of the API common to these systems
I used to run one on-prem from DDN. Another good one is Nutanix. There are many out there.

If you have a big use case and you really understand your needs, it's very doable.

You can use a different vendor any time, it's all S3 compatible. You just don't get the same performance and billing.
Except this has uniform billing, security, locality, monitoring, tools, etc
I did mention vendor lock in?