[framecowbird]

But isn't this exactly what the author is demonstrating? You can use the higher language of Python with its garbage collection, and since the critical parts are in C anyway, it ends up being competitive to the pure C implementation.

[wokwokwok]

"competitive"

Right, so let me get this straight, the argument is:

You can implement an arbitrary system in python and it is 'fast enough' to be usable and 'better' in terms of 1) speed to develop, 2) lower complexity (ie. lower LoC, easier to maintain) and 3) the garbage collection doesn't matter?

I strongly disagree.

I've worked in python for a long time, and it's a great glue language... but, it's not suitable for implementing high performance systems. Flat out.

Not. Suitable.

If the system you're developing is a mild variant on 1) something that already exists and 2) is implemented in a lower level language, then yes, python is a reasonable glue language to link together native modules.

That's why many of the machine learning frameworks use python; because it's great at allowing you to express 'high level concepts' using low level primitives.

However.

It is not suitable for implementing low level primitives; because its too slow and single threaded.

So... you might argue that this redis implementation uses enough pre-existing code that someone else has written that it is reasonably performant, but... once you go beyond the 'trivial' implementation that uses someone else code, you'll find it's really not suitable for this kind of use-case.

I love python; but this is... it's just wishful thinking.

Just because you like python, does not make python suitable for every workload.

[gomoboo]

You say that Python is unsuitable here but isn’t that subjective?

60% of the speed of Redis with its core functionality could be more than suitable for some.

> It is not suitable for implementing low level primitives; because its too slow and single threaded.

Redis was single-threaded for much of its life. That didn’t stop it from excelling.

I’ll add that I also have worked in Python predominantly. One of the things that frustrates me about it are the packages that use lower-level language bases that need compilation during a ‘pip install’. Hunting down dependencies gets old pretty fast when you were expecting ‘just Python’. For those that are ok with that and the performance hit, Python can for sure be a suitable tool for use cases that would traditionally be tackled lower down the stack.

[wokwokwok]

Not really. Python is sufficiently order-of-magnitudes slow, that it is not possible to implement pure python low level primitives.

There are no pure python low level primitives; everything is either a) wrapper, or, b) slow as hell and uses memory like a hog.

Python that wraps another language is a perfectly good way to doing things; but those low level primitives are never written in python.

...and neither are the low level primitives used in this case (—-> https://github.com/redis/hiredis).

[gomoboo]

I think we’re on the same page here with thinking Python wrapping another language is an alright way to do things.

There might be room for going lower with Python via interpreters like PyPy. Memory usage will still be high but speed will improve and you get the benefit of Python’s ease of use. For some that’s what matters most.

Personally I’m looking for a language that marries Python’s ease of composition and simple package building with typing that speeds up development in an IDE. I haven’t found that language yet. I’d be interested to hear any suggestions you have if you’ve gone down that road.

[ImprobableTruth]

That's not how it's framed though. It mentions nowhere that most of the work is actually done in C and he literally states that "The aim of this exercise is to prove that interpreted languages can be just as fast as C" which is just incredibly misleading.

Or look at the start of the readme, he claims that he wants to disprove "some of the falsehoods about performance and optimisation regarding software and interpreted languages in particular". What falsehood exactly? To me it seems he intended it to be "interpreted languages are slow", but he doesn't disprove that at all.

[eecc]

Well, 40% slower isn’t exactly what I’d call competitive...

[coldtea]

In what universe? People use Java, C#, Go, Node etc services that are 40% or more slower than equivalent C ones, and they're just fine with it...

[wokwokwok]

Yes, it’s probably fast enough to be perfectly usable.

The point here is that implementing new features will be impossible, because this relies on an existing implementation that is not in python.

It is therefore not a proof that arbitrary high performance applications can easily be written in python.

It is simply an example of how high performance applications based on existing implementations someone else had written in another language can be wrapped in python.

[coldtea]

>The point here is that implementing new features will be impossible, because this relies on an existing implementation that is not in python.

Isn't that the opposite? Implementing new features will be easier, because it uses the C backend a helper library (for parsing, eventing), so all the business logic is Python which is easier to extend.

This is also why people use embedded Python/Luc/etc in games, 3D programs, and so on.

[secondcoming]

If it was pointed out to their higher-ups that their infrastructure costs could be reduced by a sizable amount by changing language they might not be so fine with it.

[coldtea]

The last thing most higher-ups care about are infrastructure costs and optimizations.

And in many cases you're the higher up, and startup founders take decisions to use slower but more flexible technologies every day...

[tutfbhuf]

No, because you'd rely on the fact that the time critical execution paths in your python application happens to be in C. This is clearly not true in general and therefore pydis just demonstrates a special case.