[bastawhiz]

Ultimately, at least IMO, no attempt to speed up python will succeed until the issue of Python's C API is addressed. This is arguably Pypy's only major barrier: if you can't run the software on it, you're not going to use it. Pyston was arguably the most serious attempt at fast python while maintaining compatibility with the API, but DBX clearly didn't see the RoI they were hoping to.

It's looking like HPy is going to (hopefully) solve this. But finishing HPy and getting it adopted is likely to be a pretty massive undertaking.

[travisoliphant]

I think this is true. I have used the Python C-API heavily having started SciPy and NumPy and Numba. I have a pre-alpha plan for addressing the C-API by introducing EPython (a typed subset of Python for extending it). It is not usable and in idea stage only, but I welcome collaborators and funders: https://github.com/epython-dev/epython. Here is a talk that describes a bit more the vision: https://morioh.com/p/6db365736476

[seg_lol]

Interesting, I assume you are familiar with Terra, Titan and Pallene research languages?

I love the idea of typed base language to implement a higher level more flexible language while still being able to drop down for correctness and speed. Gradually dynamically typed, ;)

Another thing to look at is https://chocopy.org/ a typed subset of Python for teaching compilers courses. Might be worthwhile pinging Chocopy students and enticing them towards epython.

What is the semantic union and intersection between EPython and Chocopy?

[1] http://terralang.org/

[2] https://github.com/titan-lang/titan

[3] https://github.com/pallene-lang/pallene

[Rotareti]

This looks interesting!

I think the approach where a typed subset of Python is used to compile a fast extension module is the way forward for Python. This would leave us with a slow but dynamic high-level-variant (CPython) and typed lower-level-variant (EPython, mypyc & co) to compile performant extension modules, which you can easily import into your CPython code.

The most prominent of such projects I know of is mypyc [0], which is already used to improve performance for mypy itself and the black [1] code formatter. I think it would be interesting to see how EPython compares to mypyc.

[0] https://github.com/python/mypy/tree/master/mypyc

[1] https://github.com/psf/black/pull/1009

[sitkack]

If CPython had slowly deprecated the C API in favor of an external extension mechanism like Lua using ctypes or cffi [1], then we wouldn't be in this mess. Best time to plant a tree.

The C API is what prevents PyPy or other Python runtimes from being able to compete and interop. The community could do this, rebase Python modules with native code to cffi so that they can run in all Pythons. The C API is neither good, nor necessary and only serves to gate keep CPython's access to the rest of the Python user community.

[1] https://www.pypy.org/compat.html

[kuu]

Yeah, the current CPython code base is just too big to pretend that anything else without compatibility will be highly adopted.

It's a bit like python3 from python2, it's been so slow and painful to transition because you cannot just "drop all your code" (I'm simplifying the issue).