[eridius]

If zlib could be rewritten in Erlang to be lock-free, why not just rewrite it in C to be lock-free instead of porting it? AFAIK Erlang isn't some magical language that allows traditionally-locked data structures to become lock-free.

[PixelPusher]

Well, zlib is fairly trivial and probably not a good example due to overheads. However, an example such as a torrent server this would make much more sense. That being said, Erlang is basically a scripting language for building fault-tolerant and parallel applications.

Using C, you might be able to get parallel, but it'll be a lot of work to make it distributed and fault tolerant.

The underlying data structures have little to nothing to do with what's being said in the article.

[eridius]

I've looked at Erlang before, and I would certainly agree that it's far simpler to write a concurrent application in Erlang than it would be in C.

I'm just taking issue with the bit at the end, where they're bragging about removing a serial bottleneck by rewriting zlib in Erlang in order to remove a lock. Rewriting it in Erlang really doesn't have anything at all to do with switching to a lock-free data structure.

[PixelPusher]

Ah yeah, I had to read it a second time to realize what you meant. That's true, that it's a bad example and doesn't make much sense.

I think what they meant to say was that they parallelized the image processing mechanism of the application as a whole.

[cube13]

I'm confused where the lock that TFA is talking about is in zlib. We've used zlib in a multithreaded environment for years, and haven't had any issues with it, and as far as I can see, there isn't any mutex or semaphore usage in the source code(http://zlib.net/ ) for the library.

It sounds like this is a zlib usage issue more than anything else.

[jarrett]

One certainly could. But to obtain the same properties you get with Erlang, you'd have to reimplement some features of the Erlang VM: Lightweight threads, message passing, etc.. Erlang is opinionated. It imposes a very specific model of concurrency. If you buy into that model, the language/VM gives it to you for free. If you don't, then you pretty much can't use Erlang.

By contrast, C is a very low-level language that can do anything. You can implement any model of concurrency in C. But you'll be doing all the plumbing yourself, or using a library that does the same. Erlang's model is not the only possible way to be lock-free, and you can pursue other options in C, if you want to.

[rdtsc]

Ok so you re-write zlib. Then re-write, imagemagic, then re-write glibc, then re-write other, etc, etc.

Yes you can do it. But after a while it is like plugging wholes in a piece of swiss cheese. That is what Joe was saying you start with code that doesn't run well concurrently because concurrency was added later. It is better sometimes to start from scratch with a language that makes concurrency the default and the sequential sections are the exception.

> AFAIK Erlang isn't some magical language that allows traditionally-locked data structures to become lock-free.

There is some magic in how it has separate heaps and how it maps schedulers n:m (n cpus say 2 to 24 to m processes say 2 to 300k), how it provides concurrent garbage collection without stopping the world, how it provides hot code reloaded if that is what you need.

No it won't make coffee for you and it might not work well for a lot of tasks but it just happens to be the right tool for the right job lately as reliable concurrent back-ends becomes more important (as opposed to say single threaded desktop applications)

[klibertp]

No, it just makes lock-free and parallel programming much easier.

[eridius]

I can see it making concurrency easier, but lock-free-ness is an attribute of the data structure and the algorithms that interact with it, regardless of how easy it is to write concurrent code.

[transitorykris]

Lock-free-ness is a consequence of data being immutable in Erlang

[eridius]

You can avoid mutating data in C if you want to. People just don't, because mutation is so convenient and fast.

[stelonix]

Yes, but standard C has no way to tell the compiler a variable is immutable (immutability != constness), so unless you go non-standard (and very verbose at it) Erlang is still a better tool for the job.