Hacker News new | ask | show | jobs
by jdreaver 1626 days ago
> If you see idle is above 20 it's recommended to explore using PgBouncer.

Exploring pgbouncer when you have lots of idle connections is a great tip, but 20 idle connections feels _extremely_ low to me. I've seen postgres databases on AWS Aurora serving over 13,000 transactions per second with hundreds of idle connections (because of client side pooling with a few dozen backend clients) just fine. In fact, around that scale is when we switched _from_ pgbouncer to client side pooling to simplify our architecture, and we noticed no degradation in any major metrics.
2 comments
mulmboy 1626 days ago
Isn't AWS Aurora PostgreSQL a completely different (but compatible) product to PostgreSQL? If so it would make sense that its ability to handle many connections is just an implementation detail
link
tpetry 1625 days ago
Aurora forked PostgreSQL very early on (before 9.6) and has been improved independent and a lot has been changed. They are effectively different databases now.
link
eis_nhan 1625 days ago
That's not true at all.

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...

AWS has "only" replaced the storage layer.

link
tpetry 1625 days ago
They changed the storage layer and the query planner in some parts to work correctly with their new storage layer. So it‘s a fork and not behaving exactly identical. I have seen some different execution plans for the same model and data because of the implications pf the changed storage layer and query optimizer.
link
mulmboy 1625 days ago
Do you know of any credible source for Amazon forking postgresql? I did some googling and can certainly find lots of people saying it, but nothing authoritative from Amazon themselves.
link
tpetry 1625 days ago
AWS is basically never sharing any details, only when it‘s benefical in terms of marketing. The changes they made had not been possible with an extension in the past, and they are most probably still not fully possible without forking. But thats not problem, they are free to fork and merge upstream changes.

For MySQL any of the big users like facebook & co. Are running (heavy) forked mysql versions with changes they need.

link
parhamn 1626 days ago
Doesnt Aurora effectively have a PG bouncer in front?
link
jdreaver 1626 days ago
Nope. There is an AWS blog post [0] from September 2021 about setting up pgbouncer in front of Aurora Postgres, and that blog post references the AWS RDS Proxy service [1], but Aurora doesn't have pgbouncer in front by default. For what it's worth, we also handled hundreds of idle connections just fine on vanilla RDS postgres.

[0] https://aws.amazon.com/blogs/database/set-up-highly-availabl...

[1] https://aws.amazon.com/rds/proxy/

link