Hacker News new | ask | show | jobs
by rgrove 2801 days ago
Yeah, I really, really don't want to break things, so I've tried to be very careful about how I'm doing this.

In the shutdown announcement I committed to keeping the site running in sunset mode for at least a year. Hopefully that's plenty of time for everyone who's aware of the shutdown to migrate, but I expect there will be stragglers.

My unofficial, subject-to-change plan for dealing with that is that at the end of the sunset year, if there's still a significant amount of traffic, I'll start throttling requests to make RawGit slower. Hopefully people will notice their websites are slow and will investigate. I'll also try to notify stragglers individually by filing issues against their GitHub repos if possible.

2 comments

Instead of throttling, you could also consider doing incremental brownouts where you drop requests for the first ten minutes every hour. PyPI did this recently when they phased out TLS 1.0, which worked really well IMO.
I wish more services shuttered this way because you can’t miss it.

I once had a web host generously give me three months to pay a delinquent bill that I missed the emails for. Sadly it just meant I thought things were fine. When they finally shut my service down, my users made me aware within minutes but it was too late.

Thanks for the suggestion! This does seem potentially more effective.
IIRC Google also uses (or used) a similar system for deprecating old APIs.

Start with failing 1% of requests randomly and slowly ramp up from there.

Incremental brownouts only work well if there is mechanisms to ensure that your service's users realize that the brownout is a deprecation warning.

By default, pip doesn't show the contents of HTTP error messages [1], so users affected by the brownout would have to take extra steps (using `-v`, visiting the PyPI status page) in order to figure out what was wrong. I think it could easily appear as a networking issue or some other sort of intermittent problem.

There was also no notification of the impending blackout on python.org. [2]

[1]: https://github.com/pypa/packaging-problems/issues/130

[2]: https://lwn.net/Articles/751800/

That's above and beyond what's needed IMO, good on you for going to that length.

In the end the only thing that will fix the broken sites is to cut it off entirely.

Thanks for your project and I'm glad you're able to bring it to a successful conclusion!