[solid_fuel]

Yeah, looks like it works fine, here's an example: https://pastebin.pl/view/a10aea3d

I'll add: FLAME is probably a great addition to pythonx. While a NIF can crash the node it is executed on, FLAME calls are executed on other nodes by default. So a crash here would only hard-crash processes on the same node (FLAME lets you group calls so that a flame node can have many being executed on it at any time).

Errors bubble back up to the calling process (and crash it by default but can be handled explicitly), so managing and retrying failed calls is easy.

[jarpineh]

Well this seems nice and easy. Thank you for the example. There's local, Kubernetes and fly.io support for FLAME (that I found after short search). I envision running main Erlang VM on a light weight server continuously, and starting beefier machines for Python tasks as needed.

[abrookewood]

That's a really good option - neatly sidesteps the risk of a NIF crash with practically no extra code.

[solid_fuel]

You do still need some infrastructure - the Flame LocalBackend is for dev mostly and I'm pretty sure it just runs in the same vm as the parent.

But yeah if you're doing ML tasks it makes a lot of sense to farm those out to beefier or GPU-equipped nodes anyway so at that point it's just a natural synergy, AND you get the crash isolation.