[gaius]

Lisp isn't popular because it's harder to read. Even if it's more terse and far more elegant, the mental tax of trying to understand another person's code is far higher than C

I am deeply suspicious of explanations that boil down to, Lisp isn't popular because the average programmer isn't smart enough. Let me tell you, anyone smart enough to use C++ in anger is smart enough to use any language.

The real world is highly competitive, and organizations that think they can win by using niche languages, do. There's APL at Morgan Stanley, Ocaml at Jane Street Capital, K at Lehman's (OK, maybe that's not so good an example), F#, Haskell, blah blah. Lisp isn't even on the radar, anywhere. What does that tell you?

[yummyfajitas]

Mathematica is widely used in the financial sector. Don't let the square brackets, prefix notation and fugly symbolic shortcuts fool you: Mathematica is lisp.

[gaius]

Well, yes, you could argue that Python is Lisp too, but my point stands: if Lisp, as in, CL, really was a huge productivity boost, organizations that have demonstrated their willingness to adopt niche languages where well-suited to their problem domain would have adopted it, but that isn't actually happening. If anything Haskell and APL are nichier (if that's even a word).

The big Lisp success stories all start from the premise of a bunch of programmers who are already hugely experienced in Lisp happening to do this project in it (e.g. Viaweb, Orbitz). But when people with 20 years experience in C complete a project successfully, it isn't blogworthy news...

[yummyfajitas]

Python is a primarily imperative/OO language. Good python code consists of writing loops and building objects. Syntactic abstraction consists primarily of looking up strings in obj.__dict__. Whenever more complicated metaprogramming is needed, strings are parsed.

    "obj__parent__name_eq"=foo

is django for

    obj.parent.name == foo.

Mathematica is a primarily functional language. The primary abstraction method is patterns (macros), which allow you to modify/build up the parse tree before evaluating it.

http://reference.wolfram.com/mathematica/guide/Patterns.html

I've programmed Python and Lisp, and have begun writing some projects of reasonable size in Mathematica. There are only a couple of major differences between Mathematica and lisp. One is syntax: f[a,b,c] vs (f a b c), as well as some infix syntactic sugar. The other is that arrays are used far more widely in Mathematica than in lisp (but that may be due to the problem domain of mathematica).

[gaius]

But you haven't answered the question. Why Mathematica and not Lisp + some libraries? Or (+ Lisp Libraries). Again, there are many organizations willing to adopt any language that will give them some extra leverage. Lisp just doesn't appear. The explanation that the average programmer just isn't smart enough is easily demolished by pointing out the C++ (and APL, and Haskell) users. So either we're all missing something or the elephant in the corner is that Lisp just isn't that useful for commercially relevant problems. Which is not to say it's a bad language, but that it's not the "secret sauce" everyone's looking for.

[yummyfajitas]

Actually, the question you are asking is "Why mathematica and not Common Lisp or Scheme + some libraries?"

Mathematica is lisp + a bunch of a math libraries + vendor support + slightly different syntax + renamed functions. The syntactic differences:

    {a b c} -> '(a b c)
    f[a,b] -> (f a b)
    a == b -> (Equals a b)
    (* comment *) -> ;;comment

Since there is no advantage to different syntax and renamed functions, people choose mathematica over other lisps (and grumble about the weird syntax).

[andrewcooke]

i agree with your main point (python isn't lisp because it doesn't have macros). but the language nerd in me can't resist commenting on your example from django. that's a very special case. it's hard to think of any other package that uses that approach. rather, the way people solve problems in python that you would solve in lisp with macros is by late binding, introspection, and higher order functions. none of these replace macros entirely, but together they can be hacked into a solution.

[lispm]

You'll get a productivity boost with good people. Those are rare and expensive.

Lisp is a bit remote from machines and their operating system: GC, data formats, ...

Lisp has many degrees of freedom. It is much harder to control. It is designed for flexibility (even at runtime).

The original Lisp was done for 'symbolic computation with recursive functions': Theorem provers, machine translation, computer algebra systems, knowledge representation, planning systems, ...

If your problem domain fits into above list, then Lisp is likely to be a good candidate language to use. Then there are related domains where Lisp has been used with success: expert systems, algorithmic composition, language implementation (compilers, ...), deductive databases. Again, Lisp may be very useful in these domains.

There are lots of domains where it does not fit that well or is mostly useless. From the domains above you can see that those are not domains that many people work in (compared to web development, database apps, accounting apps, ...). Lisp is quite useful in those domains, but they tend to be a bit remote from the current mainstream.

Viaweb and Orbitz are exceptions. Most Lisp applications are relatively unknown and they are not widely published. Huge successes were for example DART and SPIKE. DART was a logistics system for the military (it is said to have repaid all the government expenses for AI research made) and SPIKE is s scheduling system for telescopes (originally for the Hubble Space Telescope). These applications were not written because some guys knew only Lisp, but because lots of the research applications in planning and scheduling systems were written in Lisp - because in this domain these applications work with symbolic representations of plans.