[jnxx]

Just out of curiosity: Why is it possible to compile Common Lisp Code (or Scheme, or Clojure) to high-performance native or jit-compiled code, but not Python? It is said that "Python is too dynamic", but is not everything in Lisp dynamic, too?

And none of these languages is less powerful than Lisp, lack Unicode support, or whatever, so this can't be the reason.

[mytherin]

It is possible to JIT compile Python just fine. There are projects like PyPy that have been doing this for a long time [1]. The reason these alternative projects never take off is because many of Python's most used libraries are written against CPython's C API. This API is giant, and exposes all of the nitty gritty implementation details of the CPython interpreter. As a result, changing anything significant about the implementation of the interpreter means those libraries no longer work. In order to not break compatibility with the enormous amounts of packages the internals of the CPython interpreter are mostly locked in at this point with little wiggle room for large performance improvements.

The only real way out is to make Python 4 - but given the immense pain of the Python 2 -> 3 transition that seems unlikely.

[1] https://www.pypy.org

[bb88]

To be fair the 2 -> 3 upgrade path was terrible. And there wasn't a killer feature in 3 which was terrible. And the tooling around the upgrade was terrible. Basically the python devs completely botched it -- which was terrible.

So one thing of golang that is nice is that go 1.19 compiler will compile go 1.1 just fine, and people can iterate from 1.1 to 1.19 in their own time -- or not if they choose not to. It would not be that hard for golang v2 to continue to allow similar compilation of old code.

[thayne]

this hypothetical 3 -> 4 upgrade would run into a lot of the same issues.

Presumably the killer feature here is that it would be faster. Or at least have the potential to be faster because of less constraints on the c API. But for a lot of python applications, speed isn't all that important. After all, if it really needs to be fast, you probably aren't doing it in python (unless python is just a thin wrapper around a lot of c code like numpy).

And for changes to the C API, it would probably much, much harder, maybe even impossible to automate migrating libraries to the new API. The only way I could see this working well is if you had some kind of shim library between the old API and the new API, but that adds constraints on how much you can change the API, and might add additional overhead.

[pkkm]

The HPy project [0] looks like a promising way out of this.

[0] https://hpyproject.org/

[miohtama]

It’s because Python object attributes can change any time, as they are accessed dynamically. Nothing can be inlined easily. The object structure is pointer heavy.

Here is some old 2014 post:

http://jakevdp.github.io/blog/2014/05/09/why-python-is-slow/

As other commenters pointed out, some of these Python features, which are unused 99,99% time, could be sacrified for additional speedup by breaking backwards compatibility.

[pjmlp]

That common excuse doesn't fly in the face of Smalltalk, Lisp, Scheme, SELF, Prolog, JavaScript, Lua.

It is more a matter of wanting to have a JIT in the box or not.

[register]

The same applies to Common Lisp. Maybe it's because type deduction is more difficult in Python than in CL?

[hathawsh]

The demand for compiled Python hasn't been as high as the demand for other languages, so the number of people who have worked on it is much smaller than the number who have built JITs for ECMAScript and others. Python has long been fast enough for many things, and where it isn't, it's easy to call C code from CPython.

Python does have lesser-used dynamic capabilities that probably don't exist in Common Lisp. Those capabilities make it difficult to optimize arbitrary valid Python code, but most people who need a Python compiler would be happy to make adjustments.

[kmod]

Having worked on this for a while, one way that might be helpful to understand this is that Python jits (such as the one I work on, Pyston) do in fact make Python code much faster, but the fraction of the time that is spent in "Python code" is only about 20% to begin with, with the other 80% being spent in the language runtime.

For example if you write `l.sort()` where l is a list, we can make it very fast to figure out that you are calling the list_sort() C function. Unfortunately that function is quite slow because every comparison uses a dynamic multiple-dispatch resolution mechanism in order to implement Python semantics.

[JonChesterfield]

If JavaScript can be compiled effectively, and V8 strongly suggests it can, it's hard to see why python couldn't be.