Hacker News new | ask | show | jobs
by throwaway894345 1835 days ago
It's all about compatibility to be honest. PyPy has done yeoman's work and it's still lacking in compatibility for important packages (e.g., psycopg2). Performance means little if we can't use major parts of the ecosystem.
2 comments
laurencerowe 1834 days ago
A few years ago I experimented with running a Pyramid/SQLAlchemy/Postgres app with PyPy. I was able to get it to work with https://pypi.org/project/psycopg2cffi/

I didn't notice any particular speedup for this app - manipulating lots of JSON Python data structures is not really PyPy's sweet spot. Whereas for some processing of large genomic data files I saw a substantial speedup.

link
jhgb 1835 days ago
Is it preferable on PyPy to use a libpq wrapper over a native solution? As far as I can tell, even though CPython C API is in some way supported on PyPy it's really only advisable to use packages using it as a last resort for performance reasons. Aside from psycopg2cffi, which you should be able to use with PyPy, I'm pretty sure I saw other PEP 249 implementations for PostgreSQL the last time I checked (which admittedly was a few years ago).
link
throwaway894345 1835 days ago
> As far as I can tell, even though CPython C API is in some way supported on PyPy it's really only advisable to use packages using it as a last resort for performance reasons

Yes, but so much of the Python ecosystem uses the C API that the "last resort" and the "common case" are one in the same most of the time. `psycopg2cffi` IIRC that's not very well supported. It looks like it was updated last in January of 2021 but before that the last update was from 2018. This was what prevented us from using it in 2020. Moreover, there are other packages besides Postgres drivers; that's just the one that I recall running into problems with. To be clear, I want Pypy to be successful, and the project is nothing short of amazing.

> I'm pretty sure I saw other PEP 249 implementations for PostgreSQL the last time I checked (which admittedly was a few years ago).

There was a pure Python version, but it didn't seem battle-tested and there was no indication of its quality or performance. I don't want to pull a package like that into production for something as important as a database driver. It's been a couple years since I looked into it as well, so perhaps things have changed for the better in the interim.

link
stuaxo 1834 days ago
HPy looks like the best compromise in this area.

It's not too different to programming the CPython API, and will give optimum code in Pypy and CPython.

link
throwaway894345 1834 days ago
Agreed, but we need the community to migrate to it, which requires leadership from the maintainers.
link
jhgb 1835 days ago
> so much of the Python ecosystem uses the C API

Yeah, that's shame, because 1) that API is mildly awful, and 2) it really restricts implementation choices to the extent that making another implementation of the language is a problem just because of the need to fake to numerous existing C extensions that your implementation choices are the same as CPython's (when really your implementation is likely to work very differently on the inside). I guess Python people really programmed themselves into a corner here.

link