Hacker News new | ask | show | jobs
by true_religion 5517 days ago
I've gotten about a 10x speed up on numerics code where there's so much branching involved in the calculations that I can't afford to use NumPy.

As for me, the main reason I haven't moved to PyPy yet is the lack of database and messaging support.
1 comments
kingkilr 5517 days ago
Which databases? At the moment we have SQLite, Oracle (haven't tested it myself), and Postgresql. Plus whatever you can find a pure python driver for. Also, what do you mean by messaging?
link
true_religion 5517 days ago
Oh wow, I didn't realize that Postgresql was working on PyPy. I heard that Django was only tested with SQLite so I made my assumptions from then on.

By messaging, I mean something like RabbitMQ, that way I can have batch scheduling at a little bit more sophisticated grain than "run a cronjob".

link
kingkilr 5517 days ago
psycopg2 is implemented in a fork of mine: http://bitbucket.org/alex_gaynor/pypy-postgresql/ it requires compiling yourself, but works nicely (I was told by someone that this brought their script's time from 2 minutes to 8 seconds). As of last test it passes all Django tests. What's the current standard RabbitMQ lib? I didn't realize it was a c-extension (hell I've used it myself and never noticed).
link
true_religion 5516 days ago
Well the most used one is Celery. It depends on multiprocessing which blew up on me the last time I tried it in PyPy.

But.... I just tried "import multiprocessing" in PyPy 1.5 and it worked! Is this all part of the C-API compatability layer? Does that mean Cython code may soon work in PyPy too (that's my pony feature)?

RabbitMQ should work under PyPy currently then, all of its dependencies purport to be pure python. ---

Another RabbitMQ lib is Rabbitmq-c which is direct wrapping around librabbitmq-c. It ecks out extra performance vs pure python rabbitmq, but mostly it isn't needed.

link
kingkilr 5516 days ago
Nope, multiprocessing was added to the Python standard library in 2.6, our previous releases implemented python 2.5, 1.5 implements 2.7, so it now includes multiprocessing.
link
asksol 5515 days ago
That's good news! Guess it's time to remove the mechanism in Celery that disables the multiprocessing pool when running under PyPy.
link
peterhunt 5516 days ago
There's also MySQL via PyMySQL
link
piranha 5516 days ago
MySQLdb also works: https://bitbucket.org/pypy/compatibility/wiki/mysql-python

I just compiled it today and it works.

link