[mrbald]

Problem of getting the concurrency right is very much identical to the problem of getting the system design right.

In both cases it takes a lot of hard work to learn how to not produce "dead-borns" or "genetically pre-disposed".

In both cases the course of "least resistance" (using intuition instead of logical reasoning) used by many as the default will most likely fail you. To develop an intuition of what is easy and what is hard with both system design and concurrency design is a matter of years of hard work of trial and error.

The actual question is why the complexity is so counterintuitive. I think this is to do with the Dunning-Kruger effect (the less you know the more confident you are).

In both cases the solution seems to be to keep beginners away from this and to have the same consistent, peer reviewed, approach across the system, and a process of improving it in a structured way. And of cause prefer higher level abstractions -> computation graph, pipeline, event loop, etc. over just hacking an ad-hoc Frankenstein out of low level primitives (which seem totally "simpler" if you ask a beginner).