Hacker News new | ask | show | jobs
by ekump 2052 days ago
This is a great write-up! I wrote a far less thorough blog post on the same topic a few months ago: https://vericred.github.io/2020/08/10/migrating-to-aurora.ht....

Overall, we're happy with our decision to migrate to Aurora. We feel it offers better performance, observability and scalability compared to heroku postgres. To echo some other comments:

1. Jesse at Heroku is great. I had never setup my own postgres instance before, and his guidance helped me a lot.

2. We used DMS, and it was ok, not great. Dealing with LOBs was a bit of a pain. In order to keep the migration time to a minimum we setup multiple DMS tasks and grouped tables into those tasks based on their max LOB size. Most of our tables didn't have any LOB columns and they migrated quickly.

3. The ability to fast clone a database is nice. We can create a copy of our production database in about five minutes for use in non-prod environments. This would take a couple of hours in heroku.

4. The Aurora IO costs are ridiculous. Are there any good tools to see what is exactly causing so much I/O?

4 comments

DMS under the covers is another tool. The Postgres support for it lags pretty well and is mostly and after thought. It's much better to try to leverage something more native, either the WAL or if on a more modern version take advantage of logical replication.

For coming off Heroku DMS isn't going to work cause you don't have a replication slot, so your only approach is wal-e, or maybe wal-g?

Fast cloning is one nice feature for sure of Aurora, but when you go into Aurora you're starting to deviate more and more from native Postgres. Fast cloning can definitely be done one without Aurora. We're working on bringing our ability on Crunchy Bridge down to a few minutes similar to Aurora, while still being 100% Postgres. Not there yet, but suspect we're a few months away.

Hey, regarding 4. we came across this but didn't find anything from the article particularly helpful for us:

https://aws.amazon.com/blogs/database/reducing-aurora-postgr...

(we bounced to normal RDS postgres and reduced our costs a lot while keeping performance similar, check out my other comments on this thread for details).

& with RDS postgres, in a lot of cases, you're probably o.k. with gp2 vs the much more expensive io1 provisioned IOPs storage unless you have a fairly business database (100s-1000s qps)

The main difference is 95% performance SLA on gp2 vs >99 on io1

(You get 3x IOP per GB on gp2)

io1 also has a slight better performance than gp2
> 4. The Aurora IO costs are ridiculous. Are there any good tools to see what is exactly causing so much I/O?

I don't use Aurora so I don't know what is available but pg_stat_statements does have I/O statistics so you can review it manually or use a tool like pgAnalyze (not sure if it supports Aurora) to see what queries are I/O intensive.

Not sure about cost and availability but Performance Insights for RDS postgres will show waits on various resources (disk, CPU) and is fairly easy to use. If it's like RDS postgres you can setup logging and use something like pgbadger (you would need to adjust the parameter group logging options to capture the right details)