Hacker News new | ask | show | jobs
by bddicken 14 days ago
Spreading requests out across hundreds, thousands, and in some cases even more is precisely what is done in the industry for big databases! Good examples:

cashapp: https://code.cash.app/planetscale-metal github: https://github.blog/engineering/infrastructure/partitioning-... etsy: https://www.etsy.com/codeascraft/migrating-etsyas-database-s...

These companies could not realistically operate off a single database server.

2 comments

The cashapp post is actually working towards GPs point. It explains that networked storage MySQL was unreliable and expensive. They migrated to $fancytool which offers a db instance with attached local nvme.

While cashapp may actually need sharding, there are so many companies who are overpaying for shitty performing network storage databases.

Not to mention you are subject to cloud provider networking and compute allocation code. They change it. Big slow network storage DB suddenly gets even worse and you don’t have leverage to have it fixed on their end.

> These companies could not realistically operate off a single database server.

I want to see it fail first. I suggest their org chart has more to do with their architecture than database performance.

Even so you’re in the category I said. Hundreds of expensive engineers maintain this stuff.

> I want to see it fail first.

your lack of experience at scale doesn't really invalidate this architecture

You’re selling a database technology without understanding ACID or transactions. You are the one who is naive and inexperienced.

Your sense of scale is entirely artificial and cargo cult. And you’re committed to selling that problem.

Have you tried maxing out the server yet? Got a write up about it?

You should read through those articles.
Thanks for sharing. They validated exactly what I said was of concern:

- loss of atomicity and transactions.

- tons of custom tooling to work around limitation of custom sql (knowing which operations are allowed and consistent).

- GitHub was able to operate for almost a decade on a single MySQL instance, and it’s not clear they really hard to change.

Etsy said this was a huge problem and took 5 years and thousands of PRs to resolve.