Hacker News new | ask | show | jobs
by bhupesh 817 days ago
Soul of erlang got me hooked to Elixir recently, trying to get my hands dirty as well.

Other than distributed/concurrent system use-cases, could you share what kind of products are best when built with elixir/erlang compared to easier to write languages like Go, for example.

4 comments

Anything that spends most of its time waiting on IO, and doesn't do a great deal of number crunching: i.e. any kind of server talking over a network socket.

Go, and any other language, lowers the barrier to running concurrent code, but there is much more to concurrent servers than concurrency: fault tolerance, isolation, shared state management, instrumentation, introspection, clustering, process migration. The BEAM and its ecosystem gives you all of that out of the box.

Also, the BEAM offers an immutable, functional environment. Data races are impossible, which are the biggest pain and source of heisenbugs in any kind of system with > 1 concurrent thread. This is huge. You can model your entire system as concurrent processes without ever having to deal with concurrency issues. You only ever have to think in "single-threaded" mode.

In other words, Erlang and Elixir push the capabilities of Kubernetes down into the language itself.
Its somewhat painful to watch modern languages and ecosystem reinvent (mostly in an inferior manner) things that Erlang (and Lisp) had since so long ago.

Talk about under-appreciated.

Virding's First Rule of Programming (2008):

  Any sufficiently complicated concurrent program in another language 
  contains an ad hoc informally-specified bug-ridden slow implementation 
  of half of Erlang.
https://rvirding.blogspot.com/2008/01/virdings-first-rule-of...
> Data races are impossible

As far as the application code goes, but most systems have databases which opens you up to all kinds of race conditions. Does Elixir help in that case?

My experience with databases in Erlang was using the pattern where one process recieves a high level message and does the database work was best. If you can split the queue in a sensible way, that gives you concurrency; for example if all the requests address a single user, you can hash users into as many buckets as you need for concurrency; if the individual requests take a long time, it might make more sense to keep a scoreboard of if a user has a process working for it, and send further requests there, otherwise to an idle process (or spawn).

You do need to manage this somehow, but the building blocks are there. Of course, the building blocks for transactionless database access are there too.

Modern relation databases are not a concern at all. You get race conditions if the developer has only a passing knowledge of SQL, because otherwise that's a basically solved problem, with the existence of transactions, isolation modes, etc. If all else fails, your query throws an error, but it does not corrupt your database.

Much more common instead are race conditions on local storage, external APIs and traditional memory races when using unsophisticated languages.

There’s a lot to learn about transaction isolation levels. I’d bet 90% of developers don’t know anything about them.

Also using serializable isn’t free.

Races and deadlocks are certainly possible in pure Erlang/Elixir - no databases or global state required. There is no magic bullet.

For example, if you use blocking RPCs between two processes, they can deadlock waiting for each other's responses. Try implementing Dining Philosophers, you will learn a lot.

The answer is not to use blocking RPC, but that will challenge your brain topology. It takes some time to be able to lower yourself into the hot bath of full asynchrony, but when you do, it feels very very good.

I characterize it as the co-problem (in the sense of the dual of a problem). Distributing a problem, starting processes and pushing messages is easy. But it may be hard to know when you're finished. Everything is easy, but termination is difficult. It's the Erlang/Elixir Halting Problem.

It is not often mentioned as a strong feature for concurrent programming, but Erlang/Elixir have timeouts built into the language. And when you add OTP supervisor restarts, you can avoid some common programming mistakes through random evasion - don't do this by design, but it does help resilience.

Races within an elixir application are still possible it is just data races that are prevented. For example if you have a bank account in an elixir process nothing will stop you from implementing a withdrawal as 2 external operations: read balance, write balance which is inherently racy but not a data race.
What language and programming style did you use prior to using Elixir the most? Have you been able to build systems of similar or even larger size as previous projects in other languages, and what was that experience like (i.e. how did Elixir make things better)?
I've been doing this job for 20 years. I am fluent in C, Go, Rust, Python, as well as Elixir, and for my clients I have written concurrent systems running in production in all of these languages. I know how painful and complex the problem space is.

My opinion is not that rare, so instead of listing my credentials, you can search and find many others that have found this platform to be a great fit, simply because it has been designed to solve this very problem since 1986 when everybody else was focusing on single core, isolated systems.

Sounds good, you make it sound like that Elixir is now your de-facto, even for non-distributed systems projects?
Servers are 99% of the code I write, so yes I choose Elixir, though for very conservative clients and small projects I use Go.

For everything else, which is not a lot, there's Rust, Scheme, Lisp and many other fun languages to explore. My focus these days is on my business rather than consulting, so I have a lot of freedom.

How do you manage to convince employers of such language choices? (Do you need to convince?) And what kind of jobs or positions are that? I would love to use my skills like that, instead of building CRUD in Python, not really being able to apply my skillset, but employers are not ready to make the smallest leap it seems.
Why specifically Elixir? Have you tried other languages that run on BEAM?
Easier to write by whom? I ask because to me, Go has one of the ugliest syntaxes I've seen. On the other hand Elixir is very easy to write.
I use both on a regular basis for many years now (and some others), and I'd say it depends.

Elixir matches the way my brain thinks somehow, with (kind of) pure functions transforming data step by step, and when you can break down some task in such a way its only a simple set of pipelines (|>) its really great and feels great. It is also exceptionally cool if you spawn Agents to hold some global state where multiple processes talk with it, plus the integrated stateful introspection/debugging is just chefs kiss. In the context of pg's "Blub Paradox" Elixir is an acceptable Lisp (not homoiconic, but with modern tooling). You can solve very complicated problems in very clean ways. Also often underrated: Phoenix/LiveView is probably the best escape hatch out of JS frontend hell alltogether, and leads to better outcomes (performance, scalability, sanity, maintainability, ...) compared to JS frameworks.

On the other hand, sometimes I have a very dirty real life thing I want to achieve. Like doing some Unix stuff, interacting with some ugly APIs, implementing an given imperative algorithm to brute force a problem quickly within reasonable constraints, manipulate some image file, automate some adhoc outlook365 process, ... you name it. The weakness of Go (extremely simple/plain, verbosity, boilerplate, ...) here is actually the strength of it in these cases, but took me a while to realize. In Go, I don't even care anymore to make anything "elegant" (which is very tempting in Elixir!), but write the absolute straightforward series of steps in brutal directness, including nested loops and very long functions. This leads to rapid dirty work solving, and has the added benefit of trivial distribution (cross-compile to a single self contained binary) to other folks that don't have dev dependencies installed. Also I rely solely and the compiler/linter for this type of code and have zero tests for it (I am not going to mock the filesystem interactions and all that for basically a better ad hoc shell script).

So for the big/complex/scalable projects, I think Elixir/Phoenix is just perfect in terms of a "web stack", only a few rough edges left in iE the docs for beginners looking into LiveView. But for the small ad-hoc stuff or in situations where I can "brute force" my way through an ad hoc problem, Go is it.

Just my 2 cents.

> Easier to write by whom?

Most people? I have no love lost for Go, but most (if not all) programmers have experience with imperative code, making it very easy to learn, even if the syntax is ugly. Elix is from a different paradigm, so you have to learn that in addition to the syntax.

Not sure how the 'best products' constraint is supposed to be interpreted, but Elixir is a more flexible tool for developers than Golang. It has a decent REPL, can be used for scripting, and so on.

I haven't used Golang for things like binary protocols so I can't really compare, but Elixir or Erlang would be a good fit since they're very good for expressing grammars and fundamentally treat strings as byte sequences.

If pattern matching helps you express your problem domain succinctly they're also a good fit. Same goes for macros. My impression is that Golang commonly requires quite verbose or complex code compared to Elixir.

I expect raw number crunching performance to be better in Golang, but BEAM processes are very lightweight so it might win on either performance or developer ergonomics if the task can be solved in parallel.

“Easier to write” has to be bait, right? That’s entirely subjective. Someone out there might think Assembly is easiest to write.