[troad]

Interesting, thank you!

I suppose what draws me to Lisp is that insight people say it gives them on programming. I already do much of my programming in functional style, so I'm trying to discover what it is about Lisp that's so beloved above and beyond that - I'm gathering it's a mix of recursion and the pleasantness of being able to get 'inside' the program, so to speak, with a REPL?

I must also admit that I tend to run into a bit of a roadblock over Lisp's apparent view that programming is, or should be, or should look like, maths. I cut my teeth on assembly, so for me programming isn't maths, but giving instructions to silicon, where that silicon is only somewhat loosely based on maths. It tends to make me bounce off Lisp resources which by Chapter 2 are trying to show the advantages of Lisp by implementing some arcane algorithm with tail-end recursion.* But I'm very open to being persuaded I'm missing the bigger picture here, hence my ongoing effort to grok Lisp.

(*Isn't tail-end recursion just an obfuscated goto?)

[cess11]

You might already be aware, but there is a DISASSEMBLE function in the CL spec: http://clhs.lisp.se/Body/f_disass.htm

The details are implementation and platform dependent, but on e.g. SBCL someone who understands assembly could use this to dig into what the compiler does and tune their functions.

I was also drawn in on the promise of insight, but I'm not so sure that's what I got out of it. What keeps me hooked is more the ease with which I can study somewhat advanced programming and computer science topics. There has been aha-moments for sure, like when many moons ago it clicked how object and closure can be considered very, very similar and serve pretty much the same purpose in an application. But it's the unhinged amount of power and flexiblity that keeps me interested.

Give me three days and I would most likely fail horribly at inventing a concurrency library in Java even though it's one of the languages that pays my bills, but with Common Lisp or Racket I would probably have something to show. As someone who hasn't spent any time studying these things at uni (my subjects were theology and law) I find these languages and the tooling they provide awesome. It's not uncommon that I prototype in them and then transfer parts of it back to the algolians, which these days usually have somewhat primitive or clumsy implementations of parts of the functional languages.

I think the reason why tail call optimisation crops up in introductory material is because it makes succinct recursive functions viable in practice. Without it the application would explode on sufficiently large inputs, while TCO allows streaming data of unknown, theoretically unlimited, size. Things like while and for are kind of special, somewhat limited, cases of recursion, and getting fluent with recursive functions means you can craft your own looping structures that fit the problem precisely. Though in CL you also have the LOOP macro, which is a small programming language in itself.

[sourcepluck]

"Algolian" is a lovely word. Does it come from somewhere, or was it yourself off the cuff?

[cess11]

'C-like language' has irked me for decades, since C was one of the first languages I learned and most languages that expression refers to are nothing like C, so when I came across lispers referring to Algol-like or Algol-descendants I took it a step further.

A web search tells me it's already in use in Star Trek.

[Tevo]

>recursion

I think one of the reasons recursion is often emphasized in relation to Lisp is because one of Lisp's core data structures, the linked list, can be defined inductively, and thus lends itself well to transformations expressed recursively (since they follow the structure of the data to the letter). But recursion in itself isn't something particularly special. Though it is more general than loops, and so it is nice to have some grasp on it, and how looping and iteration relate to each other, and it is often easier to reason about a problem in terms of a base case and a recursive case rather than a loop, at a higher level you will usually come to find bare recursion mostly counterproductive. You want to abstract it out, such that you can then compose your data transformations out of higher level operations which you can pick and match at will, APL-style. Think reductions, onto which you build mappings and filters and groupings and scans and whichever odd transformations one could devise, at which point recursion isn't much more than an implementation detail. This is about collections, but anything inductive would follow a similar pattern. Most functional languages will edge you towards the latter, and I find Lisp won't particularly, unless you actively seek it out (though Clojure encourages it most explicitly, if you consider that a Lisp).

>the pleasantness of being able to get 'inside' the program

Indeed, that's one of the things makes Common Lisp in specific particularly great (and it is something other contemporary dialects seem to miss, to varying degrees). It lets you sit within your program and sculpt it from the inside, in a Smalltalk sort of way, and the whole language is designed towards that. Pervasive late-binding means redefining mostly anything takes effect pretty much immediately, not having to bother recompiling or reloading anything else depending on it. The object system specifies things such as class redefinitions and instance morphing and dependencies and so on, such that you can start with a simple class definition, then go on to to interactively add or remove slots, or play with the inheritance chain, and have all of the existing instances just do the right thing, most of the time. Many provided functions that let you poke and prod the state of your image don't make much sense outside of an interactive environment.

There is a point to be made about abstraction, maths, and giving instructions to silicon (and metaprogramming!), but I'll have to pass for now. I apologize if this is too rambly, I tend to get verbose when tired.

[lispm]

> I think one of the reasons recursion is often emphasized in relation to Lisp is because one of Lisp's core data structures, the linked list, can be defined inductively

Lisp was used in computer science education to teach "recursion". We are not talking about software engineering, but learning new ways to think about programming. That can be seen in SICP, which is not a Lisp/Scheme text, but a computer science education book, teaching students ways to think, from the basics upwards.

Personally I would not use recursion in programs everywhaere, unless the recursive solution is somewhat easier to think about. Typically I would use a higher order function or some extended loop construct.

[troad]

Not at all too rambly, very interesting, thank you. Your answer makes intuitive sense to me; I'll ponder over it.

[pfdietz]

It's important to distinguish between Common Lisp and Scheme. The two approaches have diverged considerably, with different emphasis. The aspects you describe in your third paragraph there are more Scheme than Common Lisp.

[lispm]

There are a bunch of things to learn from Lisp:

* list processing -> model data as lists and process those

* list processing applied to Lisp -> model programs as lists and process those -> EVAL and COMPILE

* EVAL, the interpreter as a Lisp program

* write programs to process programs -> code generators, macros, ...

* write programs in a more declarative way -> a code generator transforms the description into working code -> embedded domain specific language

* interactive software development -> bottom up programming, prototyping, interactive error handling, evolving programs, ...

and so on...

The pioneering things of Lisp from the end 50s / early 60s: list processing, automatic memory management (garbage collection), symbol expressions, programming with recursive procedures, higher order procedures, interactive development with a Read Eval Print Loop, the EVAL interpreter for Lisp in Lisp, the compiler for Lisp in Lisp, native code generation and code loading, saving/starting program state (the "image"), macros for code transformations, embedded languages, ...

That's was a lot of stuff, which has found its way into many languages and is now a part of what many people use. Example: Garbage Collection now is naturally a part of infrastructure, like .net or languages like Java and JavaScript. It had its roots in Lisp, because the need arose to process dynamic lists in complex programs, getting rid of the burden of manual memory management. Lisp got a mark & sweep garbage collector. That's why we say Lisp is not invented but discovered.

Similar the first Lisp source interpreter. John McCarthy came up with the idea of EVAL, but thought it only to be a mathematical idea. His team picked up the idea and implemented it. The result was the first Lisp source interpreter. Alan Kay said about this: "Yes, that was the big revelation to me when I was in graduate school—when I finally understood that the half page of code on the bottom of page 13 of the Lisp 1.5 manual was Lisp in itself. These were “Maxwell’s Equations of Software!. EVAL is the E in REPL.

Then Lisp had s-expressions (symbol expressions -> nested lists of "atoms"), which could be read (R) and printed.

This is the "REP" part of the REPL. Looping it was easy, then.

People then hooked up Lisp to early terminals. In 1963 an 17 year old kid ( https://de.wikipedia.org/wiki/L_Peter_Deutsch ) wrote a Lisp interpreter and attached it to a terminal: the interactive REPL.

A really good, but large, book to teach the larger picture of Lisp programming is PAIP, Paradigms of Artificial Intelligence Programming, Case Studies in Common Lisp by Peter Norvig ( -> https://github.com/norvig/paip-lisp ).

A beginner/mid-level book, for people with some programming experience, on the practical side is: PCL, Practical Common Lisp by Peter Seibel ( -> https://gigamonkeys.com/book/ )

Both are available online at no cost.

[MarceColl]

Common Lisp is not a functional programming language in most current definition of the word. It's as procedural as they come, then libraries on top build other paradigms.

Scheme tends to approach things more math-like. While common lisp is less academic and more practical.