[mpweiher]

"...take advantage of the processing power of multicore processors"

Step 1: stop using Python.

"You can have a second core when you know how to use one"

Now don't get me wrong, Python is a perfectly fine language for lots of things, but not for taking optimal advantage of the CPU.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Relative performance compared to C is somewhere between an order of magnitude or two slower. Considering how much harder and more error-prone multi-core is, maybe first try a fast sequential solution.

[TickleSteve]

You're absolutely right (but you're probably gonna get some downvotes for saying that).

The ratio between the most-performant parallel framework and the least on Python will be a factor of (guessing) 1.5.

The ratio between a CPU-bound algorithm written in C and one in Python will be of the order of 10000 (again guessing as it's application-dependent).

Where is your time most profitably spent?

[devxpy]

Wild Guess: You're not really a python programmer, are you?

Just curious...

[TickleSteve]

Yes, I am... ( and C/C++ )

But I use the right tool for the job. Python is a great tool, but not for performance (Applies to all dynamic, interpreted languages TBH).

[devxpy]

> C/C++

Got it!

[auggierose]

Yeah. Recently switched some Blender Python algorithms I wrote to Swift/Metal, and the speedup was somewhere between 1000 and 1000000 depending on the algorithm.

[stevesimmons]

Speedups of that magnitude suggest the original Python approach was particularly inefficient...

[auggierose]

Not going to dispute that. If I spend time optimising code, I might do it as well in an environment like Swift/Metal instead of Python.

[targafarian]

Yeah, properly written Python is at extreme worst O(1000) times slower than speeding it up code with a Numpy/Numba/c/Fortran/etc. implementation. Brute-force loopy code in Python I've seen is 100x slower than the compiled alternatives. So I agree, these extreme numbers are the sign of writing the worst possible Python implementation of a thing and saying Python sucks.

[kilon]

Who would have guessed that compiled, static, non-dynamic, hardware accelerated code would be a ton more performant than runtime, highly dynamic, garbage collected and very powerful code that is not hardware accelerated.

[auggierose]

not sure though what you mean by "very powerful code" in that context ;-)

[igouy]

> Considering how much harder and more error-prone multi-core is, maybe first try a fast sequential solution.

Most of the Python programs referenced on that benchmarks game webpage are in-fact using multi-core ?

[eternauta3k]

Why? I can run my Python plotting script with multiprocessing in one of the blade servers at work and get the job done quickly. All without translating a big bunch of code to C.