[trapper]

Surely if lisp makes a programmer N times more efficient then it would be easy to study scientifically. Have there been any studies comparing productivity?

Why is it that in RAD application competitions, "mainstream" languages often win?

Also, if it were truly n times better the library disparity wouldn't exist; libraries should be n times faster to make or duplicate.

I'm asking for evidence here: how can something be so much better and have such a paucity of evidence for it's efficacy? If there really is such scientific evidence for lisps effectiveness I would love to read it & and distribute it to my colleagues in charge of large development teams.

[randallsquared]

libraries should be n times faster to make or duplicate

Oh, they often are... and that's why nearly everyone just builds their own. Ease of building it yourself compared to learning someone else's system is a major reason Lisp is languishing, in my opinion.

[silentbicycle]

Oddly, it can also be a downside. A language that makes reinventing the wheel too easy means that people can spend more time saying, "Psh, I could do that in a couple hours" than sharing. Then, when people want to know what the library is to use for parsing HTTP headers (or whatever), everybody and their nephew shows up to suggest their own half-assed framework. It doesn't inspire confidence.

Take that to its logical extent, and ... wait, which Lisp implementation do you use again? Mine only works with SBCL, crap, nevermind.

When somebody has a really thoroughly debugged C library, it can be like huddling everyone together by a fire for warmth in the tundra. If you don't share, man, that's just cruel.

[xpaulbettsx]

I agree - I really think that when it comes down to it, Lisp isn't popular because it's harder to read. Even if it's more terse and far more elegant, the mental tax of trying to understand another person's code is far higher than C, where there's only "the dumb way" to do something.

If people can't understand the code, they can't contribute, so you don't end up with the network effect of F/OSS contributions that make a language popular (note that I say popular, not better)

[nocman]

As a person who started learning Lisp after years and years of using mostly mainstream languages, I have to at least partially disagree. I will say this, however: Lisp is, at first, difficult to read for many people. But I think it is mainly difficult to read for people who are used to reading programs written in languages with Algol-like syntax (C, C++, Java, Perl, etc). If you've been writing programs for 5, 10, or 15 or more years mostly in languages that all have that common ancestry, it is understandable that your brain will (at first) work against you when reading something like Lisp. However, after using it for a reasonable period of time, I found that it became much easier to read. And now I don't find it any more difficult to read than any of the mainstream languages that I have used in the past.

I think that, in fairness to Lisp, if you take two people who have never programmed in their life, and if each of the two has a reasonable and approximately equal amount of skill and potential for being a decent developer, and taking those two individuals, you teach one to develop in Lisp, and one in, say Java. Given that situation, I don't think the Lisp developer will find his code more difficult to read really at any stage of his learning (more difficult that is, than the Java developer finds his code to read at a similar stage of learning).

It seems to me, from every testimony I've read from developers who have come to Lisp after extensive careers using other more popular languages, that there is a "breaking in" period where it takes some time to get used to things. However, anyone I've read about who put in any reasonable effort to learn a Lisp (or a language related to Lisp, like Scheme) seems to repeat what others like him or her have said -- you get used to the syntax after a while, and then at the very least it no longer bothers you or gets in your way.

[Jach]

http://img264.imageshack.us/img264/1397/lispnd7.png

[joe_the_user]

I program in C++/etc but also in SQL. SQL is terse logical language. Everyone may has SQL but I think given constraints of being a logical language SQL is far more readable than lisp - but I've only read bits of lisp online.

[jules]

I find CSS to be far more readable than SQL.

[silentbicycle]

I'm not clear what point you're making. CSS isn't a logical language, and they don't really have enough in common to compare them. (And FWIW, I find SQL clearer than Nahuatl.)

[gaius]

Lisp isn't popular because it's harder to read. Even if it's more terse and far more elegant, the mental tax of trying to understand another person's code is far higher than C

I am deeply suspicious of explanations that boil down to, Lisp isn't popular because the average programmer isn't smart enough. Let me tell you, anyone smart enough to use C++ in anger is smart enough to use any language.

The real world is highly competitive, and organizations that think they can win by using niche languages, do. There's APL at Morgan Stanley, Ocaml at Jane Street Capital, K at Lehman's (OK, maybe that's not so good an example), F#, Haskell, blah blah. Lisp isn't even on the radar, anywhere. What does that tell you?

[yummyfajitas]

Mathematica is widely used in the financial sector. Don't let the square brackets, prefix notation and fugly symbolic shortcuts fool you: Mathematica is lisp.

[gaius]

Well, yes, you could argue that Python is Lisp too, but my point stands: if Lisp, as in, CL, really was a huge productivity boost, organizations that have demonstrated their willingness to adopt niche languages where well-suited to their problem domain would have adopted it, but that isn't actually happening. If anything Haskell and APL are nichier (if that's even a word).

The big Lisp success stories all start from the premise of a bunch of programmers who are already hugely experienced in Lisp happening to do this project in it (e.g. Viaweb, Orbitz). But when people with 20 years experience in C complete a project successfully, it isn't blogworthy news...

[yummyfajitas]

Python is a primarily imperative/OO language. Good python code consists of writing loops and building objects. Syntactic abstraction consists primarily of looking up strings in obj.__dict__. Whenever more complicated metaprogramming is needed, strings are parsed.

    "obj__parent__name_eq"=foo

is django for

    obj.parent.name == foo.

Mathematica is a primarily functional language. The primary abstraction method is patterns (macros), which allow you to modify/build up the parse tree before evaluating it.

http://reference.wolfram.com/mathematica/guide/Patterns.html

I've programmed Python and Lisp, and have begun writing some projects of reasonable size in Mathematica. There are only a couple of major differences between Mathematica and lisp. One is syntax: f[a,b,c] vs (f a b c), as well as some infix syntactic sugar. The other is that arrays are used far more widely in Mathematica than in lisp (but that may be due to the problem domain of mathematica).

[lispm]

You'll get a productivity boost with good people. Those are rare and expensive.

Lisp is a bit remote from machines and their operating system: GC, data formats, ...

Lisp has many degrees of freedom. It is much harder to control. It is designed for flexibility (even at runtime).

The original Lisp was done for 'symbolic computation with recursive functions': Theorem provers, machine translation, computer algebra systems, knowledge representation, planning systems, ...

If your problem domain fits into above list, then Lisp is likely to be a good candidate language to use. Then there are related domains where Lisp has been used with success: expert systems, algorithmic composition, language implementation (compilers, ...), deductive databases. Again, Lisp may be very useful in these domains.

There are lots of domains where it does not fit that well or is mostly useless. From the domains above you can see that those are not domains that many people work in (compared to web development, database apps, accounting apps, ...). Lisp is quite useful in those domains, but they tend to be a bit remote from the current mainstream.

Viaweb and Orbitz are exceptions. Most Lisp applications are relatively unknown and they are not widely published. Huge successes were for example DART and SPIKE. DART was a logistics system for the military (it is said to have repaid all the government expenses for AI research made) and SPIKE is s scheduling system for telescopes (originally for the Hubble Space Telescope). These applications were not written because some guys knew only Lisp, but because lots of the research applications in planning and scheduling systems were written in Lisp - because in this domain these applications work with symbolic representations of plans.

[10ren]

Maybe that generalizes, to say that it's the code that lisp doesn't turn into "Fungible Cogs", and so it can't be used by others in a standardized way.

Standardization seems to be anathema to the individuals who are attracted to lisp's power and flexibility. If lisp ever achieved code fungibility, lispers would hate it.

[10ren]

6. And the LORD said, Behold, the people is one, and they have all one language; and this they begin to do: and now nothing will be restrained from them, which they have imagined to do.

7. Go to, let us go down, and there confound their language, that they may not understand one another's speech.

Genesis 11:6-7 http://www.omniglot.com/babel/english.htm#kj

[camccann]

And here I thought that was talking about perl...

[jacquesm]

I think a bigger factor is the amount of prior knowledge required before you can be productive.

In C (and c-like languages) that barrier is relatively low, a handful of concepts and a very limited subset of library modules and you can go a long long way.

[gaius]

Indeed. Lisp programmers with 20 years experience are more productive than Java programmers with 2 years experience. That doesn't say anything about Lisp vs Java (or, not nearly as much as some would claim it does).

[trapper]

This is a testable hypothesis due to the number of popular open source projects around we can draw from.

[rbanffy]

Ease of building new stuff brings fragmentation:

http://news.ycombinator.com/item?id=1013716

[joe_the_user]

Ease of building it yourself compared to learning someone else's system is a major reason Lisp is languishing, in my opinion.

Oh, that why Perl is languishing too. Well, the problem is how hard it is to learn someone else's system in Perl. If Lisp has that problem, well, there you are.

[randallsquared]

I don't think Lisp has that problem. Rather, it has the "problem" that it's easier to build the 20% you need of that library than to find the library, understand enough to do the bit you need in the face of that inevitable impedance mismatch between your abstractions and the library author's, figure out how including it impacts the rest of your system including the dependencies...

In other languages, this is just something everyone has to live with, and because of that, libraries face strong selection pressure to be one or more of simple, fast, standard, etc. There is no particular pressure like that in Lisp, since there's a much smaller domain where it's easy to understand why you need a thing, and troublesome to build it. To a much greater degree than mainstream languages, then, there are almost as many "language + standard library" sets as there are Lisp developers. You can see this same effect at work in Python web frameworks, I think.

[gnosis]

Perl is definitely not languishing. It just gets a bad rap from fans of other languages that are competing with it (particularly from the fans of Python and Ruby, the two upstarts trying to take its place).

Let's look at the number of jobs advertised for Perl, Python, Ruby, and Lisp over the last 5 years:

http://www.indeed.com/jobtrends?q=perl%2Cpython%2Cruby%2Clis...

The number of Perl jobs is more than those of Python and Ruby put together. Lisp has so few jobs it doesn't even really show up on the chart.

Then there's Perl's CPAN archive, with over 17,000 modules, with about 30 modules updated or added per day. That much development is a sign of an active, alive community.

So the rumors of Perl's demise are greatly exaggerated.

[pg]

Curiously, it's inherently impossible to compare what I think matters most for languages: solving new problems. To compare two languages you have to have people solve the same problem, and that means it has to be a predefined one.

[gruseom]

I agree. Predefined problems are one way people compare languages; code snippet comparisons are the other. Neither touches the important thing, which is how languages differ as a medium for thought. Different languages give rise to different ideas, which give rise to different programs.

The other problem with studying this question formally is that it is so psychologically rooted that people almost invariably "find" that their favorite language turns out the best.

[trapper]

This is what statistics are for, and is exactly the same situation that many other disciplines find themselves in where treatment a influences treatment b.

[bOR_]

Not the easiests of problems though. Do you know what statistics you would apply?

IQ measurements seem to have a similar problem: doing test a, influences your score on test b, because you get more used to the problems presented.. but doing test a and b just marginally increases your general intelligence, which is what you are trying to measure, while it might have a big influence on someone's score in these tests.

With IQ tests, you can expect that doing test c till z will eventually saturate someone.. but that might only work because you generally do not get the answers. In coding, you do get the answer back (whether your code compiles/works or not), so eventually both programmers will just be limited by the speed with which they type.

[trapper]

>Not the easiests of problems though. Do you know what statistics you would apply?

There are many ways. Off the top of my head:

- to focus on speed of development you could use a practice or warm-up

> provide each with a practice in another language

> provide a sample solution in pseudocode in both functional and imperative style

- To focus on speed of problem solving you could use topcoder data + research of participants programming language backgrounds.

The important thing is that if the effect is large it's easy to spot. According to some, the difference is an order of magnitude. If that's true, it will be trivial to spot.

[10ren]

Perhaps one source of data is YC startups. What's the success rate of lisp vs. non-lisp?

Yes, of course, many things contribute to success other than language choice. That's my point.

[asciilifeform]

> Surely if lisp makes a programmer N times more efficient then it would be easy to study scientifically. Have there been any studies comparing productivity?

I began to write a reply to your question, but decided that it deserved its own post:

http://www.loper-os.org/?p=85

[plinkplonk]

"I began to write a reply to your question, but decided that it deserved its own post"

That isn't really a reply to the question posed. Just more ranting and raving.

Sample: "Yes, my dear bean counters, you can measure productivity. I would no more ask you to stop in your attempts at its measurement than I would ask mosquitoes to stop sucking blood. You can measure productivity – even of Lisp; even of political philosophies. You will simply need to secure a very large bag of beans – one deep enough to hold a bean for every twist and turn of a century of tinkering, politicking, and everything else associated with the messy business of successfully thinking new thoughts."

The best interpretation of this drivel is that you want people to accept your claims of superior productivity just on your say so but not ask for any evidence because asking you back up your unsubstantiated claims of superiority would be "bean counting".

Forget the scientific studies. There is another kind of "evidence" in programming - write code that is evidently superior. The best way to show that (say) Linux is an "inferior" operating system is to build one better. Not talk about making one sometime in the misty future. Build. Then talk.

Endless rants aren't as good. As silentbicycle says elsewhere

"I just take issue with how some particularly vocal Lispers have been sitting on the sidelines going, "Our language is THE BEST LANGUAGE" for decades. ("Sick of being a blub programmer and working in Blub? Try our Language For Smart People.") They're so used to being smug because their language was thirty years ahead of its time, fifty years ago."

[camccann]

The best interpretation of this drivel is that you want people to accept your claims of superior productivity just on your say so but not ask for any evidence because asking you back up your unsubstantiated claims of superiority would be "bean counting".

The obvious interpretation is that he's saying something like "the usual definition of productivity is flawed and myopic, and gets in the way of realizing the power of Lisp". The paragraph you quoted (implied as a representative sample) is indeed pretty much content-free, because it was a gratuitous flourish at the end.

Of course there is indeed a lot of assertion couched in flowery language there, but that's not the same as "because I say so and if you disagree you're a poopyhead" which is what you seem to have gotten from it. You also seemed to miss the part where he argued that multiple major concepts in modern software originated on the Lisp Machine, which would seem to more than qualify as "building something better".

I'm assuming you're not naive enough to think that "better" equates to "successful", which is another issue entirely.

[plinkplonk]

"The obvious interpretation is that he's saying something like "the usual definition of productivity is flawed and myopic, and gets in the way of realizing the power of Lisp". "

Just the fact of him sayng something like that doesn't make it true. More logic, supporting arguments and evidence is called for to support such claims. Besides He isn't offering any (lucid) alternative definition. Just rant after rant , over a period of years. Which makes the whole post (and link to it on HN) very content free (imo, feel free to disagree and vote accordingly).

Yes , sure Lisp is cool. There I said it. But hey Linux is cool too. So is C. So are a lot of things.

[wendroid]

> The best way to show that (say) Linux is an "inferior" operating system is to build one better.

One of the most valued metrics of "better" is "how does it fit into the ecosystem". Plan 9 is clearly superior to Linux. It has cleaner interfaces, one person can understand it, lot's of things are unified that were disparate etc. etc. Everyone who uses both extensively berates Linux at every turn.

Mention it in more general spaces as HN, Reddit, Slashdot etc. and you will be told it is a failure, flop, doesn't have program X so it's useless, etc. etc.

Popularity means nothing

http://www.tiobe.com/content/paperinfo/tpci/index.html

The previous winners of the "Language of the Year" award are shown below.

    Year	Winner
    2008 	C
    2007 	Python
    2006 	Ruby
    2005 	Java
    2004 	PHP
    2003 	C++

[plinkplonk]

"Plan 9 is clearly superior to Linux. It has cleaner interfaces, one person can understand it, lot's of things are unified that were disparate etc. etc. "

I have no argument with this,

"Everyone who uses both extensively berates Linux at every turn."

I seriously doubt this. Please provide any evidence you have.

If the makers of plan 9 were to say "Linux is crap" at least they have the credibility to say so. It is noteworthy that they would never say something so stupid. Linux (and yes Plan 9) aren't the acme of perfection. But neither are the "lisp os" es. Raising ancient lisp systems to some unreachable pedestal of perfection , especially by people who've never seen or used on and have just read about them, is dubious

This is just a blogger with lots of crummy blog posts and no code. At best he is incoherent and at worst deluded.

To repeat my point was not that Linux is the best OS ever. But for me (or anyone else) to go around screaming that everything that is popular is all delusion is pointless (and strictly imo) shouldn't be paid any attention.

"The previous winners of the "Language of the Year" award are shown below.

    Year	Winner
    2008 	C
    2007 	Python
    2006 	Ruby
    2005 	Java
    2004 	PHP
    2003 	C++"

And the point is? Most of them are fine languages and worthy of "langauge of the year" at least when measured in terms of popularity.

"Popularity means nothing"

Popularity is not everything but it sure isn't "nothing" either.

[wendroid]

> Please provide any evidence you have.

Only the power of my own ear at Plan9 conferences from people who know a damn sight more about such things than I. I could pull various mailing list posts out, such as the big argument of Rob Pike with Linus but what's the point.

> If the makers of plan 9 were to say "Linux is crap" at least they have the credibility to say so. It is noteworthy that they would never say something so stupid.

From the Plan9 fortune file : "Linux: written by amateurs for amateurs." - D. Presotto

> And the point is?

That the technical merit of the language doesn't determine it's popularity. So choosing what is the "better language" is rendered pointless.

>Popularity is not everything but it sure isn't "nothing" either.

Yes, I read that back after the edit time ran out and wanted to change it. Popularity and userbase isn't the correct metric for quality.

[plinkplonk]

""Linux: written by amateurs for amateurs." - D. Presotto"

This is (in a strict sense) true and is not the same as "linux is crap". Linus was certainly an amateur when he wrote the first version of the kernel in the sense that was the first OS kernel he wrote) and many people who contribute are not tenured professors(or whatever your definition of "professional" happens to be). The fact remains that an "amateur" teenager in Finland ran rings around the "professionals".

"That the technical merit of the language doesn't determine it's popularity. So choosing what is the "better language" is rendered pointless."

I agree. But that is the point isn't it? You can't choose "the better language" anyway. The mythical Lisp OS isn't really "better" either unless you've made up your mind up front. So why go on and on for years (the blogger not you)about "everything popular is crap and Lisp is the ultimate leet cool thing forever and forever and anyone who doesn't accept this is an idiot" instead of focussing on actually writing some code?

[trapper]

I refer you to:

http://news.ycombinator.com/item?id=1016762

What's wrong with that as evidence for lisps superiority?

[jules]

What's wrong is that they measured the performance of Lisp programmers vs Java/C++ programmers. This is fine but you're not comparing the merits of the programming language. The Lisp programmers would win even if they were forced to program in Java.

[ams6110]

Maybe it's not that lisp makes any programmer more productive, but that it offers the anecdotal "order of magnitude more productive" programmer features that Java, being designed for the "interchangeable cog" programmer, does not.

[jcapote]

Paul Graham writes about this extensively in http://www.paulgraham.com/avg.html

[trapper]

It's computer science not computer woo. I'm asking for evidence not anecdote :)

Surely, there has to be at least some evidence in it's 50 years? Especially given the claimed huge effect size.

[sjs]

It's simply impossible to isolate and control all of the important variables involved in developing any software of any significant size. There's a reason for the lack of productivity data for programming in general.

[trapper]

One impossible study:

http://news.ycombinator.com/item?id=1016762

[sjs]

Do you think if it were repeated today with a recent version of the JDK Java development time might be less? Probably not as low as Lisp but things have improved in Java land since 1.2.

That synopsis of a study is nice but I'm still not convinced all the important variables were isolated.

[trapper]

Eclipse + java6 would make a marked difference, agreed.

[Herring]

That sounds like a lisp proponent's problem, quoting stats he can't back up.

[sjs]

There are a dozen reasons the data would be bogus but I only need to state the most obvious: you can't use the same person in 2 comparable tests because the tasks would be either too trivial or too similar; likewise you cannot use different people in groups for the tests because people are too different.

Smart and dumb people have succeeded with all kinds of languages on all kinds of platforms that are widely viewed as amazing and terrible and everywhere in between, on all kinds of projects.

[naveensundar]

trapper, this study might be in the flavor of what you are seeking. http://www.flownet.com/gat/papers/lisp-java.pdf (2000)

[YuriNiyazov]

There has never been a rigorous scientific study done, no, but there's plenty of anecdotal evidence, and, to quote Paul Krugman's tongue-in-cheek title, "the plural of anecdote is data"

[trapper]

That's the same explanation psychics give. If the effect size is large (as is claimed), it should be an extremely simple study. So why has no one done it?

Why don't the lispers organize a RAD competition between lisp and all-comers, surely if lisp is that much more productive you would see the top ten places being lisp dialects.

Would you agree?

[tel]

Extremely simple in principle, obscenely difficult in implementation. The real difficulty is the confounding variables. Being unable to accurately measure or predict things like level of language familiarity, programming experience, domain knowledge, or intelligence means that it'd be impossible to tie any visible effects to something as simple as language choice.

So, no, I wouldn't agree.

The best study I could think of off hand would be a randomized study of programming productivity (measured somehow) of programming naïve high schoolers. Each subject gets an incentive to learn some language over the course of X weeks and then program something. The sample size would have to be magnificent even to prove a large effect because you'd need to demonstrate control for intelligence/drive and regressing out math scores (or anything similar) would not be convincing really. At the end, you'd still not be able to talk too much about working productivity, but you might have a chance of talking about... anything at all.

Actually, this is an extraordinarily difficult study I think.

[gridspy]

Perhaps high-school or college students are poor evaluators of a language. I am not only interested in how productive you are after 1 year of using a language, I'm also interested how productive you are after 10 years using that language, or with 10 years of legacy code.

[tensor]

Well, I'm not sure a RAD competition would necessarily be considered scientific. For one, the sample size is small. It also ignores properties such as maintainability, assuming "first to finish" is your only criteria. Finally, it's necessarily a fairly small application I would think.

As for studies, a quick search on google came up with this: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.117...

Which concludes that "Haskell prototype took significantly less time to develop and was considerably more concise and easier to understand than the corresponding prototypes written in several different imperative languages, including Ada and C++."

I couldn't find much more in a short time. Perhaps my search terms, "functional programming imperative productivity" were inappropriate.

[trapper]

Now that is interesting. I personally agree with the hypothesis that lisp is a step forward, but often wonder why it's an anecdotal argument than one that is data backed.

This paper although makes me think they compared programmers not languages, as the imperative language programmers didn't have any functional experience. It would be interesting to see a comparison between programmers of roughly equal competency.

[tensor]

Mostly it's due to a lack of good, controlled, data. The ideal study would need to be of a large and complex system that was recreated several times in several languages. You'd need to verify each system behaved the same, and that the teams on each version were nearly equally competent in their respective languages. Even that would be poor, since it would be a single data point. Perhaps a particular language was simply good at that problem!

Despite that complaint, the above is nearly impossible to do in practice. So you have to settle for smaller problems, which can be done in several months perhaps. Even at that, it would take some very difficult collaboration, and would still be a single data point.

Further complicating issues is the academic world vs. industry. Again, ideally you'd want to do the test in industry, with an appropriate team of programmers and managers. But industry doesn't want to waste time with such academic matters, nor does it want to open it's code for analysis.

In other words, it's a very difficult problem to address, despite being seemingly simple on the surface.

[tel]

This paper is another interesting anecdote, unfortunately. They're not really claiming anything quantitatively.

Also, though it's not explicit, I feel it's obvious that the researchers had a prior bias toward functional languages.

[camccann]

A competition to solve a toy problem wouldn't necessarily say much, though. So the question is, how big of a problem would you need, and where are you going to find enough programmers (of reasonable and roughly similar competence) who don't have anything better to do than spend multiple hours participating in a glorified programming language pissing match?

In other words, the reason why not is probably "because organizing things is hard work".

[trapper]

If the effect size is large (i.e. the magnitude of difference between the means of lisp vs non-lisp programmers) the size of problem should be irrelevant.

Many thousands of programmers spend hours each week in pissing matches online. Why is it none of them feel like truly putting their money where their mouth is?

[camccann]

If the effect size is large (i.e. the magnitude of difference between the means of lisp vs non-lisp programmers) the size of problem should be irrelevant.

Writing a "Hello, world" is going to take about the same amount of time in any language that isn't a turing tarpit. At issue is generally some sort of asymptotic advantage, not a guarantee of "language X will always be superior every time always". In fact, for any pair of one easy and one hard problem, I'm pretty confident I could find (or invent, if necessary) two languages that each beat the other for solving one of the problems.

The claimed benefits of a more expressive language generally involve things like better management of application complexity, or greater ability to modify and restructure late in the project. Hammering out a program of a scale such that the programmer can keep an accurate model of the entire thing in mind at one time, and that is only written and never modified, would be more a measure of the programmer's typing speed and the sophistication of the IDE used than anything else.

[prodigal_erik]

If a problem is so trivial that creating a domain-specific language isn't worth the effort, the best solution will be equally ugly in any language and Lisp loses its advantages.