Okay, interesting. I guess I've seen it before, so I didn't think it was Ruby-specific, but I also haven't had to do that kind of programming in a very super long time.
It's fairly common to define malloc and free wrappers called xmalloc and xfree. The usual idiomatic thing to do is to have xmalloc check the return value of malloc and terminate the program if it ever fails, though this is traditionally done in application code rather than a library. I doubt libc would ever add an xmalloc function because it'd break all the code that defined its own xmalloc.
Replacing malloc/calloc/free etc an application-specific xmalloc/xcalloc/xfree is seemingly a fairly common pattern. There's one example in libiberty[0] (-liberty, get it?) where, for instance, the return value of malloc is checked for NULL and the program terminates if so.
Publib then? I'm finding this when I search: http://man.cx/xfree(3) http://man.cx/publib(3)
Also, are you sure? I see xmalloc in glibc all over the internet...