[jacquesm]

Erlang excels at the 'hard' parts of the problem space having to do with complex distributed ultra reliable systems, the 'number crunching' bits are offloaded to specialist languages because the underlying language model that Erlang uses does not lend itself for the kind of optimization that JS, Go, PHP etc have available to them simply because their runtime model is so completely different.

The biggest issue from what I know about how Erlang works internally is not so much the fact that Erlang could not in principle be made fast (and indeed, BeamASM will if it makes it to mainstream become a good step in the right direction), but that the whole way in which Erlang schedules its threads is super ineffecient in terms of cache use and besides the byte code interpreter relies on its ability to keep track of the number of reductions that it has done to determine when a thread has had enough cycles and we need to move on.

This means there will always be a fairly hard upper limit as to how far you can optimize Erlang byte code, it is at its core a cooperative multi tasking operating system inside a user process.

One of the more elegant ways to do such interop is to isolate your number crunching code to a separate process group working their way through a queue one unit of work at the time that way you can use all of the Erlang goodies and still get very good performance.

[derefr]

> One of the more elegant ways to do such interop is to isolate your number crunching code to a separate process group

Or, if you want to maximize perf inside some hot section of code (say, a game-engine renderer) while also retaining Erlang semantics for how it interacts with the rest of the system, you can write your code as a "threaded NIF" — a native thread (in C or Rust or whatever) that sends messages back over to the arbitrary Erlang processes in its address-space (not just its owner process!) as it works.

[alberth]

What concerns me is I see people talk about using NIFs like no big deal.

BUT, they come at the cost of potentially bringing down the entire Erlang runtime. People use Erlang, more times than not, because they want a fail-safe runtime ... so recommending NIFs shouldn't be taken lightly.

Just my 2 cents.

[derefr]

> People use Erlang, more times than not, because they want a fail-safe runtime

I disagree; 99% of people running Erlang runtimes are running them in use-cases where it'd be fine if any individual node crashed and got restarted.

(E.g. running a Phoenix web-app as a K8s replicated Deployment, with no durable internal state, only the mesh-replicated Phoenix.Presence data that can be recovered from peers after node restart; where, for users, even having a long-running websocket is just an optimization over long-polling, and so it's no skin off the clients' backs if said connection gets dropped once in a blue moon, requiring them to reconnect to another node in the cluster.)

I agree that you should only be using NIFs after you've exhausted the other options — "C nodes" being my favorite of those, personally.

But downplaying NIFs ignores a whole other segment of use-cases, where you're not adding native code to Erlang, but rather you're starting with a "core" of native code (e.g. an HFT trading engine), and then building an app by wrapping that native code in Erlang, where the code presents itself to Erlang as a NIF.

If efficient shared-address-space high-throughput interaction between your native "core" and your higher-level "glue" is the whole point, then Erlang+NIFs is a pretty great way to get a bunch of advantages without losing much. It's a lot better than what you get from the alternatives — where those alternatives are "loading the native core as an FFI module in some other dynamic language runtime."

[the_only_law]

> the 'number crunching' bits are offloaded to specialist languages because the underlying language model that Erlang uses does not lend itself for the kind of optimization that JS, Go, PHP etc have available to them simply because their runtime model is so completely different.

Yeah I love Erlang, but often reach for something else when I need to be able to express and optimize compute bound problems.

[jacquesm]

In a way it makes perfect sense. When I look into my toolbox I see a whole pile of different saws, each of which has a specific usecase, metal, wood, living wood, trees, joinery, figure work, glass+ etc. The one-tool-to-fit-all-usecases programming language hasn't been invented yet as far as I can see.

+ technically, probably more of a grinder because the blade is covered with diamond dust, but you use it like a saw.

[MrWiffles]

> The one-tool-to-fit-all-usecases programming language hasn't been invented yet as far as I can see.

Well, just like you can use a screwdriver to pound a nail into the wall (flip it handle-first), you could use C or Rust or Go or Java or...nearly anything to build...nearly anything if you really _forced_ it hard enough, had infinite time, expert labor and budget, but...

...there are better tools for the job of pounding a nail into the wall than a screwdriver! ;-)

In other words, I totally agree with you. Sure, you could build, oh I dunno, say a compute-intensive application entirely in, say, Ruby, but you'd need to throw some ungodly expensive hardware at it. Makes a lot more sense to use the right tool for the job!