Hacker News new | ask | show | jobs
by tedsuo 3341 days ago
The most common problem is that code executing at that point can hang in unexpected ways, preventing shutdown. It's a real bummer when that happens. I've even seen it happen with logging written the wrong way, where the code attempts to flush the logs to ensure they are written... and hangs.

Meanwhile, the crazy code that panicked is still running it's other goroutines - remember, you called recover! - so maybe now the webserver still has an open port and is allowing your users to access whatever strange state is left inside... gonzo things really can happen if you let a corrupted program stay on rather than shutting it down immediately. Even if nothing bad is happening, you still are out of commission for that entire period.

The point is, murphy's law always comes into play. So if we're talking about production best practices, consider that "most likely fine" means "definitely not fine at scale over time". Just make sure that whatever you're doing during shutdown can't block.

1 comments

Would you suggest the same course of action for any language which uses exceptions? Because there is nothing special about the stack unwinding you get with panic/recover.

I'm seriously wondering if you ran into any trouble with recovering panics in production, because that would imply all Java, C#, Python, JS and Ruby server code in production which is happily catching and logging exceptions in the main request handler is constantly running into corrupted state.

That's not an accurate comparison. The issue is not the stack unwinding, but the root cause of the panic. In all of those other languages, exceptions are used for normal error handling in addition to "fatal" events. In Go, panics, if used at all, are generally used for fatal runtime errors (memory, segfault, etc), or in place of assert, and indicates a fatal, unrecoverable flaw in your program.

At every job I've worked at it was generally discouraged to handle MemoryError exceptions in python and OutOfMemoryError exceptions in Java. It's safer to assume there is nothing you can do on such occasions, because sometimes (usually the worst possible time) there isn't.

Another relevant Java example might be VirtualMachineError (superclass of OutOfMemoryError) and the wonderfully named subclass UnknownError(!).