[geocar]

If I am from another planet, and say I don't know why programs are so big and slow and buggy, and the most complicated program you see I've produced is a glorified calculator, it's too easy to be patronising and say well, that's because you haven't done anything complicated.

However if I then show you a programming language, a database engine (similar in capability to SQL but around 1000x faster), a graphical desktop with icons, mouse, editors, filesystems, ISR, and so on- and it's still under 400 lines of C code, then maybe it's easier to have this conversation that I think we need to have: That maybe all you programmers have simply been doing it wrong, and there's something fundamentally wrong with the way you and everyone else programs computers.

I think most programmers take so much offence from the thesis: that everyone programs wrongly (or badly) and that a fundamental shift in the way we program could make programming much better, that it's been difficult to actually work on this problem. Just look at all the people who are complaining about the text editor being difficult to read without saying I can't read this, and I want to get better.

And I think it should be obvious: You've built bridges for thousands of years, so you expect you're pretty good at it now, and yet there are still improvements in bridgemaking today; why do you expect programming to be any different? Why do you have "best practices" for programming if you've only been doing it for a few decades and don't really know how to do it yet?

Maybe it's much easier to think we're working on giving non-programmers the ability to program, but we're not: we're trying to make programming suck less.

[MichaelGG]

I'm hesitantly excited about this language, after reading this article. The power increase I've gained going from an "OO" imperative mindset to functional has been huge. The simple volume of code from other styles just boggles my mind. I can't understand why people prefer their verbose code with so much edifice. Making another leap like that sounds very promising.

At the same time, it does sound slightly off. What's the catch? In the text editor example, how capable is it? Often, such claims rely on a trick, like saying "<textarea/>" is a text editor in 11 characters. Or the tiny Haskell quicksort that's actually rather inefficient.

I think I'll try playing with it. How long should a Logo interpreter in K be?

[geocar]

I wanted to be able to type K and press a button and get the result, so the first bit of k code I wrote was the following:

    np::*((l$[=/k;(*i),0;i]),j)_a;ci:{{J y;x:3:. x;J y;kx x;K(y,j)}[np;j]}

When you type !10 and press control-I it prints the result where the cursor is and selects it. This way you can press backspace to delete the output and change the code easily.

How much code should that take? Probably less than that (I didn't know about bin yet), but I think that's part of the learning process.

I think a minimal logo would only take a couple lines.

[minor_nitwit]

that's awesome. It's a one line REPL.

[beagle3]

I think Douglas Adams' SEP[0] field is a good description of this. Most people ignore what doesn't fit with their logic - doubly so if fitting it in would show them that they have been wrong/wasted money/wasted time in the past. (The Upton Sinclair quote about "it's difficult for someone to understand something when their salary depends on not understanding it" is also relevant).

It's that way with religion, paradigms in medicine, and even chemistry/crystallography (see e.g. Shechtman vs Pauling[1]). It's human nature.

It is also worth pointing out that while this specific version of K is new, K itself is 20, and it is mostly purified APL, itself over 60 years old. People have been ignoring this since forever (APL found stellar success in the OR community in the 70s and 80s on its own, in trading floors in the 80s and 90s thanks to Arthur Whitney, but has since mostly disappeared)

[0] http://en.wikipedia.org/wiki/Somebody_Else%27s_Problem#Ficti...

[1] http://www.theguardian.com/science/2013/jan/06/dan-shechtman...

[selimthegrim]

Nothing encapsulates PNAS' worthlessness as a journal more than that fiasco.

[jwatte]

If the SQL is I/O bound, the kdb version cannot be 1000x faster. And if it's not I/O bound, you're not pushing it right. (Nite: compare columnar SQL here.) K can't make your I/O subsystem faster, and the new OS probably doesn't natively support 99.9% of existing high performance hardware.

[icsa]

The SQL was not I/O bound, per se - that is database/implementation-specific. The benchmarks were evaluated on "hot" data with plenty of RAM available (like any good production system).

Some databases don't manage that situation as well as they should. I.e. - they were developed in an era of small RAM & large disk.

> And if it's not I/O bound, you're not pushing it right. That's exactly the point. k is "pushing it right" while the others don't do nearly as well.

> the new OS probably doesn't natively support 99.9% of existing high performance hardware.

At <500 LOC of ANSI C there's not much to port to new hardware, given a decent C compiler.

[jeffreyrogers]

>And I think it should be obvious: You've built bridges for thousands of years, so you expect you're pretty good at it now, and yet there are still improvements in bridgemaking today

Right, and I'm not saying programming won't improve. It obviously has. No one wants to write a web app in C++ or assembly. What I'm specifically taking issue with is the idea that there is some sort of monumental change out there that is going to enable us to do... what exactly?... well no one can really answer that, but the claim is that it is big and exciting and will change everything.

[avmich]

VPRI (vpri.org) was already mentioned here. The goal of Alan Kay, as I understand it, to reduce complexity of computer systems - by making them smaller. k shows you potential results with this approach - doing things differently, you can have small and functional systems, so they could be easier to understand while remaining feature-rich.

[geocar]

When I can fit my program on the screen, I don't make any mistakes.

I think this is true of most programmers.

While "hello world" type programs tend to be the pedagogical example, kOS demonstrates that the complexity of such a one-screen low-defect program is much higher than people previously thought.

[MichaelGG]

Not being facetious, but does this extend to using larger screens, multiple screens, wider columns or multiple windows, and smaller fonts? Or was one screen just meant as an estimate for 200 lines or so?

[geocar]

I think it is the act of scrolling or window-switching or head-moving that is the cost. To that end, I find using a MBA screen to be perfectly adequate for programming because it fits entirely in my field of view.

My screen is about 55 lines tall and maybe 170 characters wide. I sometimes shrink or increase the font slightly to make programs fit on screen.

[ddoolin]

I'm just a regular web and app developer, but would it be at all possible to write a layer on top of k that makes it at least more readable while retaining most of the speed advantage?

To me, part of being an engineer is not taking offense when being told there are potentially better ways to do things, but at the same time I can understand why some people jump to that kind of statement since at first glance it seems like a regression of sorts, especially when the industry has a continuing history of trying to lower the entry barrier. Even though underneath the programming concepts may be revolutionary, people may also be quick to balk at it when it's in such a form (hence the first question).

[avmich]

Industry does try to lower the barrier to entry. At the same time it strives to reach ever-higher tops. APL family of languages have a high barrier to entry - which is of course a problem - but even remaining novice in the language you can be routinely more efficient working with APL than working with another language. So - that high barrier to entry is a disadvantage, but for that you can have pretty high levels of efficiency in the same language going from newbie to intermediate to advanced levels.

[geocar]

I think this "readable" bit is the theory behind q which you can download from kx.com; Some people think it is more readable. q/kdb+ supports websockets and has a built-in http server so it is very possible to make web apps with it.

However I think there is value in the dense coding style that has nothing to do with it's performance: it's that it reduces bugs and helps you think about the problem.

[ddoolin]

You've sparked my interest, for sure. I'd love to pull it down and give it a whirl. Hell, one could conceivably write those things with K as well.

Thanks for the perspective. Good luck to you all with kOS and others.

[icsa]

Having studied the History of Mathematics, I think that the same discussion/argument must have occurred in Italy when transitioning from Roman to Arabic numerals.

Would you rather do:

IXDCCCLVI * VIIDCCCXLIX

or

9856 * 7849 ?

[xorcist]

> similar in capability to SQL but around 1000x faster

If it's really similar in capabilities, why not strap an SQL parser on it and sell it? You can do it client-side to avoid wasting performance where it matters. Surely there's money to be made in the database business if you are an order of magnitude faster than everyone else -- let alone three.

[geocar]

You don't seem to be aware that Arthur did exactly that:

http://code.kx.com/wiki/Startingkdbplus/qlanguage http://www.kx.com/q/d/kdb+.htm http://www.kx.com/q/d/kdb+1.htm

It's a commercial product and I understand kx does pretty well.

[xorcist]

I was not. Thank you. What is the source of the performance figures?

[geocar]

http://www.listbox.com/member/archive/107499/2014/05/sort/ti...

The list is subscriber-only, and I apologise that the information isn't very easy to get at.