Hacker News new | ask | show | jobs
by bitlad 6 hours ago
PostgreSQL is good enough until it's not good enough, when you realize all the bad design decisions that were made before it hits scale. It is the decisions people make around not partitioning, HA, replication that makes it not good enough.
6 comments

These are not bad decisions they are reasonable tradeoffs at the beginning.

Requiring HA, partitioning, and replication are good problems to have.

The alternative is spending engineering time on setting all these up for a failed service with like 100 users.

It is not good to have a problem that requires a year of migration by a dedicated team (typical case if you have this 'good problem') when you could have simply spent another day or two to architect it right. You should at least have a path to scale when the time comes.
I feel like any problem that Postgres can't handle is a good problem to have. Either you've got so many customers that you're hitting sharp edges, or you're working on such an interesting problem that you're out of the domain where Postgres is helpful. That I should be so lucky
A moderate-size queue is already a problem that Postgres struggles to handle %) But fortunately such problems are rare, and relatively well-known, like, say, doubly-linked lists in Rust.

For most other problems, Postgres works well, or at least well enough.

That's fine. There are plenty of projects that don't hit that scale.
There are also plenty of projects that do. PostgreSQL is good enough for most projects. But it isn't enough for every project.
The problem is that way too many people assume they're in the latter group, when the proportions are very much the other direction
Re: bad design decisions - This can be said about any technology.
+1 - when to prematurely optimise, when not to.
Oh yeah - I remember working with a guy who absolutely insisted that we build application-level sharding into what we were working on (i.e. each primary key would have some metadata that our application logic would use to route traffic to the correct Postgres instance). Our database was not big - 10s of gigabytes - but Instagram did something similar and we needed to be able to scale. I pushed back hard, and fast forward two years and the application was decommissioned without hitting any sort of Postgres bottleneck.
Everything is "good enough until it's not good enough". That's engineering.
HA is now pretty good on Postgres.

As for scale... Just use a larger machine. This works for regular transactional data until you're at something like Amazon scale.

Edit:

Think about this, suppose that you store 1 megabyte of data for each of your customers. So if you have a million customers, it's just 1Tb. And these days, you can have a server with 10Tb RAM delivered overnight. Although you might have to sell your firstborn son (offer applies only to royal families) to fund it.

A lot of sharding/no-sql/... development happened in the late 2000-s when computers were about ~100 times less powerful than now. You _could_ get a system with 10Tb RAM in 2010, but as a specially-designed supercomputer.

128Gb of RAM is like 1.5k
For a non-ECC version. A single-module 128Gb ECC stick is about $4500. HBM is even more expensive.