Hacker News new | ask | show | jobs
by jojo2000 2103 days ago
Let me explain my line of reasoning here (been there at least three times, situation gave 3 different outcomes) :

- case 1 : need to make calculations with a specialized library in C (precision arithmetics). Build a bare FFI, later replaced with CFFI.

- case 2 : loop-intensive on very simple calculations, time-constrained development. Identified as horrible performance in python : tried numba didn't work, ended up using cython, worked really well.

- case 3 : optimize numpy-intensive routine for performance. Tried numba, didn't work. Too expensive to recode in cython. Look at numpy C/C++ interfaces, and numpy-friendly C++ alternatives. Also try to trick numpy to function better (do not ever try this, it'll give worse results). Ended up doing nothing as time to develop was the main constraint here.

If you have a lot of time and work on a small program, maybe you can spend the time to optimize. Team producing lots of complicated algorithms and no way to re-develop everything, stick to python, identify perf losses and choose wisely what you'll optimize.

Numba has progressed but is still not a "drop-in decorator" as advertized. Can even give worse performance in some cases. Nevertheless the idea is good and I praise the effort, when it's done it'll be massive !

1 comments

Numba failing to work in case 2 is extremely bizarre. Can you post that code or a similar example?

Not only should it work for case 2 - it should only require a one liner decorator add and no user specified static typing.

I’ve used numba in production for around 6 years now, and never encountered the problems you describe.