Hacker News new | ask | show | jobs
by illumen 3713 days ago
I don't think it's so cumbersome with modern python. There's first class coroutines, futures, process pools and such now. Also there are a number of queues systems in python which easily allow running failed tasks again. One project I use is basically task(func, args). That's it. Then if it fails we can have them automatically restart, or wait for review etc. This is over multiple machines too. It's only about 60 lines of code, including nice gui interfaces to manage the tasks, and deploy the whole lot to various machines.

Erlang is amazing of course. Just wanted to say that some of those patterns are now available in python and easily usable. Not common to all python code, which is where I think erlang wins out. It puts this stuff front and center, and has first class support. Looking at a random erlang code base, you'll probably see it there. Random python code bases... not so much (ok, queue systems are quite common in Django/Flask projects). It's very rare to see python greelets/eventlets in the wild for example, but generators and async stuff are becoming quite commonplace.

Also python has single dispatch now (built in, not in a third party library). Another thing which Erlang does well (pattern matching). Again, not so commonly used except in modern python shops. These combined with quickcheck for python (hypothesis), and gradual typing really have made modern python a much more happy place. Erlang deserves some big respect for spreading good ideas.

3 comments

Sure. Individual aspects of Erlang/OTP are all present in other systems. OS processes, queuing backends, light weight co-routines, Java has some code reloading too.

But it makes a difference if it is in one language/framework and built-in. It makes tracing/debugging/developing easier.

Like with Python, yeah can use a queuing subsystems and submit jobs. But that is another service to configure and manage. Can use multiprocessing (and I've done that), but can't launch 1M of sub-processes. Which now changes how you develop. It has a green-thread co-routine support via greenlet (eventlet & gevent) but those share memory and if you do any CPU intensive work will block each other and will also share the heap.

I'd say it's a stretch that Java has code reloading. Sure you can reload the bytecode, but unless you're marshaling the data to the new version like Erlang does you're just introducing more errors than fixing things.
Mutable languages fundamentally can not replicate the totality of Erlang's fail-first approach because the possibilities of partial state updates are fundamental issues. You can personally write code that is still fairly safe to use that way, but you're doing it by hand without compiler/runtime support, and you can't control your libraries.
I think it's pretty easy in Python (things like Celery), but I think the interesting thing is that for Erlang this stuff comes baked into the language, so it's even easier.

So there's all this nice fault tolerance by default, whereas in Python you have to look for it (and you can forget it)