Hacker News new | ask | show | jobs
by alienthrowaway 484 days ago
> Because it uses NIFs (natively-implemented functions), an unhandled exception in Pythonx would take down your whole OS process along with all other BEAM processes, making your supervision tree a bit worthless in that regard.

What's the Elixir equivalent if "Pythonic"? An architecture that allows a NIF to take down your entire supervision tree is the opposite of that, as it defeats a the stacks' philosophy.

The best practice for integrating Python into Elixir or Erlang would be to have an assigned genserver, or other supervision-tree element - responsible for hosting the Python NIF(s), and the design should allow for each branch or leaf of that tree to be killed/restarted safely, with no loss of state. BEAM message passing is cheap

1 comments

That's the thing though: a NIF execution isn't confined to the the BEAM process by its nature. From the Erlang docs:

> As a NIF library is dynamically linked into the emulator process, this is the fastest way of calling C-code from Erlang (alongside port drivers). Calling NIFs requires no context switches. But it is also the least safe, because a crash in a NIF brings the emulator down too. (https://www.erlang.org/doc/system/nif.html)

The emulator in this context is the BEAM VM that is running the whole application (including the supervisors).

Apparently Rustler has a way of running Rust NIFs but capturing Rust panics before they trickle down and crash the whole BEAM VM, but that seems like more of a Rust trick that Pythonx likely doesn't have.

The tl;dr is that NIFs are risky by default, and not really... Elixironic?

> The emulator in this context is the BEAM VM that is running the whole application (including the supervisors)

You are correct - one could still architect is such that the genserver hosting the NIF(s) run in a separate process/VM/computer in the same cluster since message passing is network-transparent, though inter-host messages have higher latencies.