Hacker News new | ask | show | jobs
by cguess 1400 days ago
Today I went to restore a backup from three weeks ago on a new test instance and I saw it broke. I was getting some utterly weird messages about the pgcrypto extension not being available anymore (problem since I use UUIDs as my ids).

After HOURS of messing around I found this https://devcenter.heroku.com/changelog-items/2446. None of which was communicated to anyone, and it completely breaks all old backups. There's no mitigation listed on the page, and from what I can tell no one can figure out the proper way to do this without manually dumping, manually changing all references, pushing it back up and hoping to god nothing breaks.

I find it absolutely, utterly, unacceptable to do something like this with no migration path documented. Most people (including myself) use Heroku because we're willing to pay more over being a sysadmin. I can run my own infrastructure on bare metal machines, but it's something I deeply, deeply, don't want to do. Much less manually editing 15gb SQL files.

So yes, I'm considering leaving if this isn't mitigated quickly.

1 comments

Yep. I am getting pretty good support communications on this issue in heroku support tickets (you are not?), but the fact that they haven't actually made a public announcement about it is pretty unforgiveable.

Last I heard from support, they thought the issue you are running into with backup restores would be solved maybe in the next week or two... but there is no public place to look for status or resolution, I just need to keep asking support, apparently. There are workarounds, that is there are ways to restore from those backups if you really need to... but it's confusing. Some more info here:

https://www.reddit.com/r/Heroku/comments/wgkjdf/heroku_ext_c...

Anyway, still being in the middle of dealing with this when the DNS issue happened... my opinion of heroku is definitely dropping fast... but it started out so high it's gonna take it a while to hit the ground and smash into pieces.

Heroku support answered my support ticket within a few hours (so good on them) with a canned response that pointed me to the article I had already referenced to them (not so good) and some hand waving "we're working on it." How they deploy this without figuring out a migration path first is beyond me. Someone... must have brought this up internally and been shot down I guess?

The issue they say this was to mitigate is this CVE https://www.postgresql.org/about/news/postgresql-145-138-121... which seems extraordinarily difficult to exploit (oh, and they don't address in already active databases... so....).

Very much throwing the baby out with the bathwater.

Plus the "we're shutting down all your stuff" message today makes me scrambling for another service.

> Plus the "we're shutting down all your stuff" message today makes me scrambling for another service.

Wait, what? i didn't get that one! I don't think....

Log in and look at the notice at the top. All free services go away in November (the day after Thanksgiving weekend none-the-less, so have fun there). Upgrade your Postgres and Redis or you're screwed.