I dont get this. Just because you don't have the GIL doesnt mean that your previously single threaded code is now multithreaded and stepping on itself.
From previous discussions it's my understanding the c integration is going to be the cause for the issues.
From a python perspective it wouldn't necessarily be a big change, but everything can branch out to c, and there you're going to get in trouble with shared memory.
The most significant impact of this change is that python threads can run at the same time. Before, calls to C API were essentially cooperative multithreading yield points - even if you have 100 python threads, they can't run concurrently under the GIL. 99 are blocked waiting on the GIL.
C extensions have always been able to use real threading. But now there is no GIL to synchronize with the python interpreter on ingress/egress from the extension.
No, but it means that your previous multithreaded code is no longer automatically prevented from stepping all over itself by having multiple threads accessing the same data:
GIL removal doesn't mean "make all of this lockless" it means "replace the GIL with fine-grained locking". So those problems are still solved for Python code. The three issues are the amount of work it takes to do right, the performance cost for single-threaded code, and the CAPI.
I'm simultaneously scared and enlightened seeing all these comments acting as if the GIL is some magic "makes your code thread/concurrency safe" pancea. I always saw it as a shim/hack to make cpython specifically easier to implement, not something that inherently makes code more thread safe or atomic. It's just more work to do things "the right way" across application boundaries, but from my understanding this PEP is Meta commiting to do that work.
Removing the lock creates problems in existing code in practice. This is an ecosystem that has less focus on standards and more on "CPython is the reference implementation".
What non-transparent GIL specific behavior are developers relying on exactly?
When I say GIL specific behavior, I mean "python code that specifically requires a GLOBAL interpreter lock to function properly"
Not something that simply requires atomic access or any of the garuntees that the GIL has this far provided, but like, specifically code that requires GIL like behavior above any CPython implementation details that could be implemented with more fine grained concurrency assurances?
I've seen some really cursed python in my days, like checking against `locals()` to see if a variable was defined ala JavaScript's 'foo in window' syntax (but I suppose more portable), but I can't recall anything specifically caring about a global interpreter lock (instead of what the GIL has semantically provided, which is much different)
> What non-transparent GIL specific behavior are developers relying on exactly?
They are relying on behavior of a single environment. We similarly see a lot of issues moving threaded code to a new operating system in C/C++, because the underlying implementation details (cooperative threading, m:n hybrid threading, cpu-bound scheduling, I/O and signaling behavior) will hide bugs which came from mistakes and assumptions.
finally. you don't even have to read anything to work this out -- if things like dictionary access were no longer atomic that would imply that threaded code without locks could crash the interpreter, which isn't going to happen.
> GIL removal doesn't mean "make all of this lockless"
Literally speaking, that's exactly what "removal" means. As far as I can tell, GP was wondering why there's so much discussion about replacement, since simply removing the GIL wouldn't break single-threaded code.
From a python perspective it wouldn't necessarily be a big change, but everything can branch out to c, and there you're going to get in trouble with shared memory.