Hacker News new | ask | show | jobs
by dewey 2 days ago
The point is that Postgres scales a very long way. Once you arrive at OpenAI / ChatGPT scale there's no shame in reaching for a dedicated queuing system.
1 comments

> scales a very long way

This ignores the fine print. It scales a very long way under specific circumstances with specific workloads. The more write-heavy your workload the less eloquently Postgres scales.

For OpenAI's use case you could swap Postgres with MySQL and it would scale just as well.

We are talking about Postgres as a queue here, which is heavy on tiny writes, reads and churning tables. The point is just that a RDMS like Postgres scales very far as a queue, it’s not a fight if Postgres or MySQL.

Nobody is arguing for using it for everything and forever but for most company sizes it’s perfectly fine to not reach for a dedicated queuing tool if you already have PG running.

> on tiny writes

Postgres doesn't do "tiny writes" - it writes whole pages multiple times on every update even if you're only changing 4 bytes, combined with even more writes later on when vacuuming tables. This is one of the reasons why the historical advice was to not build high-volume queues on top of Postgres and why "oh look another post about queues on Postgres" keeps soliciting comments like this.

If your DB is write heavy, tune it for writes...
You can't tune your way out of this constraint. The heap table and MVCC implementation put a hard ceiling on what can be accomplished without reaching for another tool.