[geocar]

What you're actually seeing is testimony: people saying they are seeing something amazing, and they aren't very good at explaining what they saw.

Btw: k doesn't translate to C. It's actually a quite simple interpreter. The fact that it outperforms other languages so easily should be saying more about those languages than it should be saying anything about k.

[burntsushi]

Well, does there exist a technical analysis written by someone who understands k that explains why it is so much faster than X languages?

[geocar]

As I said, I think this is the wrong question.

The real question is why is X language so slow?

This is not intended to be glib: but I do not think I can put it more simply than that. X language is slow because it uses lots of library code that it doesn't actually use (to get a friendly interface), it has a lot of redundancy (because of the wrong abstraction level), and because it wastes memory (in order to have an API that links well with others). Or it's slow because it thinks B-Trees are really cool. Or because it has the wrong intrinsics. I don't know.

But I am convinced this is the discussion we need to be having.

[burntsushi]

It's not the wrong question. You just rephrased it. And in the process of doing so, you missed the point of my question. I emphasized the word "technical" because I was trying to politely ask for evidence. Evidence should be some combination of code, benchmarks and analysis.

The code should provide isomorphic samples from the languages (or implementations of languages) that are being tested. Ideally, the code samples should be idiomatic.

Benchmarks should test the aforementioned code such that performance comparisons can be made. (With some degree of accuracy.)

Analysis should summarize and draw tempered conclusions from the code and benchmarks. Trade offs should be documented.

The Computer Language Benchmarks Game[1] is a first approximation of this. It provides the first two things but has no analysis in prose.

Does such a thing for k exist? Even if it's a very rough blog post somewhere from 5 years ago?

I ask this because there's a lot of lofty claims being made in this thread, and the closest thing to evidence I've seen is, "trust us, it's made a lot of money doing [financial things]. oh, and it can update a million records in a second." Since I don't believe that a free lunch can exist, I am naturally inclined to reach the truth of the matter. I see that k is supposed to be wicked fast---at what cost? Where are the examples? Where is the analysis documenting this?

[1] - http://benchmarksgame.alioth.debian.org/

[icsa]

/ http://shootout.alioth.debian.org

http://kparc.com/z/comp.k

[burntsushi]

Ok... Now how do I run those programs such that they are comparable to the ones on the benchmark game?

[icsa]

Make a request at kparc.com for the research version of k (k5).

If that doesn't work download q/kdb+ from kx.com (Windows/Linux/Mac OS X). It can runk4.

/ k4 implementations of the Bell Labs benchmarks are available http://kx.com/a/k/examples/bell.k

[mwcampbell]

So would k be even faster if it were compiled to machine code rather than being interpreted? Would that involve an unacceptable speed versus space trade-off? Or am I missing some crucial reason why k code has to be interpreted?

[Torn]

According to wikipedia's uncited 'performance characteristics' section of their page on K (programming language):

> The small size of the interpreter and compact syntax of the language makes it possible for K applications to fit entirely within the level 1 cache of the processor.

Sounds like the overhead's acceptable?

.

Edit: the following page (2002) says

> Even though K is an interpreted language, the source code is somewhat compiled internally, but not into abstract machine code like Java or Python. The interpreter can be invoked interactively or non-interactively, and you can also compile a source file down to a binary file for product distribution, if you wish.

http://www.kuro5hin.org/story/2002/11/14/22741/791

[smcl]

Thanks for the link to kuro5hin btw - it's a nice intro to k but also I've not been on k5 for years, I had no idea it was still going :)

[beagle3]

It would be even faster, with a sufficiently smart compiler (a feasible one, not they mythical beast[0]). E.g., the function

    sumsquares:{+/a*a*a*a:1+!x}

which sums pow(a,4) for a from 1 to x (its argument), when interpreted, allocates a vector of length x, initializes it to the numbers 0..(x-1), then allocates another vector of length(x), to which it copies the first vector while adding 1 to each element, and calls it a. Then makes another vector of length(x), puts a * a in respective elements, etc. - and finally, sums the elements of the remaining vector.

If we can learn from numpy (K equiv) -> numexpr (same work, smarter about allocations) -> numba (jit compiler, avoids allocations), we can expect x2 faster speed in both "upgrades" - about x4 overall.

If that sounds too little speedup for so much effort, you are not giving modern caches enough credit. Almost every single memory access here is in L1, as they are all sequential and perfectly predictable. So it's just 2-5 slower than doing the perfect register allocation with no memory access - but with the cost of ballooning code size (going out of L1 itself ...). Also, interpretation overhead is nearly nonexistent because operation on vectors are coded in tight cache aware C loops.

Overall in the interpreter/compiler tradeoff, K comes out far enough ahead of everything else that it did not make sense for them to invest in a sufficiently smart compiler. In fact, it is only a few months ago that they bothered using SSE/AVX vector instructions for significant speedup. It is so far ahead of the curve (for its use cases) that doing that didn't really give any marketing advantage until recently (and I'm not sure it does now).

[0] http://c2.com/cgi/wiki?SufficientlySmartCompiler

[icsa]

I don't think so. Many of K's primitives operate on arrays using stride 1 access. Being cache-friendly accounts for much of the performance gains.

In addition, K is written in ANSI C to maintain portability.