Hacker News new | ask | show | jobs
by fweimer 26 days ago
Surely you need an alternative to Box<dyn Error> for reporting memory allocation failures?!
5 comments

Anything other than panic/abort on allocation failure is outside the scope of the vast majority of programs, including anything using the standard library in Rust. I wouldn't worry about Box<dyn Error>.
You're already writing Rust in a very different style if you're writing the type of code that gracefully handles allocation failure. It's to Rust's immense credit that this type of coding is actually fairly well-supported (unlike in Go), but you're already a bit off the beaten path for stuff like error handling.
Not sure what your problem is?

If you need to handle an allocation error in the error path, then the error reporting path must abort, which means that the allocation error must be bubbled up.

There is no real solution to an allocation error inside the error path. Even if you preallocate an arena for errors, the error might be large enough that it won't fit inside the arena.

Hence the best thing you can do from that point onwards is to have an error enum with an AllocError variant that doesn't allocate. Said error won't contain any information beyond line numbers of the allocation error since you just don't have the space for it.

In the end you will basically end up with panic free code, but the error still bubbles up like regular unwinding.

So yeah you can do it, and I will do it in the future, but I personally think that the people who think this is some huge deal breaker don't understand the problem in the first place.

A &(dyn Error + 'static) should be fine for that; you don't need any allocated/variable sized data in a memory allocation failure.
stacktraces? might also be useful to know whether or not the latest allocand was a jumbo sized allocand that caused the failure?
Do you really want that data passed back down to the caller of the allocation? From the description of the failure state you'd want to log that data instead: what's the caller of the allocation going to do if you tell it it failed with a crazy size? It already knows the size, it's the one who asked for it.
So, suppose it's a rust library -- you're locking me into whatever logging system the library author chooses? Maybe I'd like to consume the relevant data at the entry point and send it to a logging system of my choice.
A Rust library likely wouldn't be returning an opaque Box<dyn Error> to begin with. Errors are part of a library's API—it's what allows consumers to handle them—so you'd define an enum of possible errors your library could produce and return that, which would be stored on the stack.
What about the data in the error payload?
usually “stdout” is good enough, wrapper/runner routes output to logserver for collation and search. who cares about formats as long as it’s reasonably structured and searchable?
it depends, if the functionality represented by the library is known to require a lot of memory (or simply allocation failures are an expected part of its operation), then it should be pretty much part of the API, probably with some tracing/diagnostics interface to get the required visibility into how much memory goes and where.

but for most libraries I on allocation failure I don't expect any fancy logging system. maybe even panic is fine.

And how do you store a stacktrace without allocating?
If you let the allocation error panic you will get your stack trace.

You can't have a stack trace on an error in the error path that failed to allocate. If you have a "jumbo sized" error and the error fails to allocate, it won't get reported. The only reporting you will get is that the error failed to allocate and this new allocation error overrides the error that failed to allocate.

if you have an OOM and want to log why, if your logging allocates it will likely fail as well. You could in principle work around this with enough effort, but "properly" handling OOM is typically much more trouble than its worth.
Any time I mention "but I would like stacktraces with my errors" I get told I'm doing it wrong.
That's because the types of errors where you want a stack trace are a relatively small subset of all possible errors.

Stack traces are only useful for errors that indicate a bug in the program, i.e. something a programmers has to respond to. It's not useful for the vast class of bugs that are a result of wrong input, wrong external state, or infrastructure issues.

Rust projects tend to favor panicking over error handling for programmer bugs (which does indeed give you a stack trace depending on environment variables), or even better encoding the invariants in the type system, but there are cases where an error coming from a library are truly, actually unexpected, so both `anyhow` and `thiserror` do provide support for attaching a stack trace in those situations.

See? You get people explaining to you that you actually don't want a stack trace because xyz.
This sounds disingenuous. They explained why the language doesn't force stack traces on all errors, and then explained how to get them if you want them.
No you just wrap it the way https://news.ycombinator.com/item?id=48267094 showed.

Obviously you need to explicitly include the line numbers since the error reporting Rust is closer to C (errno) than Java or JavaScript.

https://users.rust-lang.org/t/getting-line-numbers-with-as-i...

Just do what feels right. I personally am a big fan of Java stack traces.

The comparison here is to Go, which doesn't have any way to handle memory allocation failure AFAIK.

In Rust you can do that (mostly on nightly although not only), and yes there are alternatives for this case, but it's rare anyway.