[jgeralnik]

  > Mypy will get rid of the Global Interpreter Lock (GIL)
  >Details of the concurrency model are still undecided, though

Wait, what? There have been numerous attempts to get rid of the GIL, most of which failed miserably. A promise to get rid of the GIL without even deciding on a concurrency model seems a little premature.

[JulianWasTaken]

This is not true. There have been plenty of successful attempts to remove the GIL, including from CPython. It's not impossibly hard to do so by a long shot.

The GIL is an optimization though, and so the performance hit from removing it on single threaded code is prohibitive. You can watch any of David Beazley's talks on the GIL for more info.

[jgeralnik]

I stand corrected. Still, saying they are going to remove the GIL without addressing the challenges faced by other people who did the same - most notably the performance hit - makes me skeptical as to their ability to fulfill their promises.

[andreasvc]

Prohibitive is too strong I think. The patch to 1.4 discussed in a blog post by Beazley came with a 2x slow down, but that was an initial patch without much work on optimization. Besides that, I think 2x slowdown isn't that bad considering the language is not used for its raw efficiency anyway.

[jules]

Well, the only reason to remove it is to run multiple Python threads in parallel for efficiency...so efficiency is definitely the top concern here.

[andreasvc]

No the reason to remove it is to get parallelism from threads. Once you have parallelism, you can overcome a 2x slow down for a single thread by using more cores.

[batista]

If 99% of your codebase uses one thread, then no, a 2x slow down is not acceptable, even if you can overcome it by using more cores.

Remember, the ideas was not to have CPython and CPython-without-GIL, it was to have one unified GIL-less CPython.

For that, a 2x performance drop would not be acceptable for most code.

[andreasvc]

But 2x was just with the initial, non-optimized patch. A language like Java seems to perform quite well even with fine-grained locking, so it's possible.

And I don't see why performance is suddenly used as a deal breaker for a language primarily used for scripting and other non-CPU-bound purposes.

I speculate that the performance issue wasn't really the most important reason for rejecting the patch, but more so not wanting to deal with the complexity of maintaining the patch. It's a shame because the future is definitely going to have lots of cores, and not all problems are well-suited to multiprocessing.

[jukkaleh]

The mypy FAQ now answers this question:

We don't believe that it is practical to adapt the CPython VM incrementally for running parallel workloads efficiently. Instead, we are going to get rid of the GIL by having a new VM for running mypy code. The VM will be based on the Alore VM, but it will use compiled native code from the start (the Alore VM uses a bytecode interpeter). It will use a garbage collector instead of reference counting (CPython primarily uses reference counting), and it will also support a modified C Python extension API that will support proper multithreading without a GIL.

The CPython VM will still be used to run Python code and modules, and it will still be limited to running a single thread at a time (most of the time). Objects passed between the VMs will have to be copied or accessed using proxy objects, so there will be some performance overhead compared to native mypy modules. We plan to port commonly used, performance-sensitive Python modules to mypy and the modified C API to minimize performance bottlenecks.