[PaulHoule]

I'm going to take the opposite position that there is much less special about Lisp than people think. Like I have met so many programmers that travel around like itinerant martial artists looking for true functional programming and they never find it.

To be specific, you can work all of the examples in Graham's On Lisp in Python except for one of the last chapters where he implements continuations that really need macros -- but this is basically the async/await facility that Python already has. The other examples use macros for performance but work fine with just plain functional programming.

React with hooks is an example of that kind of system at work -- the JSX transformation is a very simple shim you can put in front of the JS compiler and the hooks themselves are the kind of trick that On Lisp teaches you how to do.

On Lisp doesn't use the kind of tree-walking macros that really are unique to Lisp. And... the techniques in the Dragon Book for writing compilers are the real magic. If looking to Lisp as an old shiny keeps you from learning how to write compilers, it is holding you back.

I think the homoiconic thing leads people astray. There's a real tension that, for performance reasons, mainstream compilers aren't extensible, for instance you might want to write a

  unless(X) {...} => if(!X) {...}

control structure in a new language and for something in Java that is really a production rule in the grammar, maybe a class to represent the unless block, and a rewriting rule that gets applied to the AST. If the compiler was designed to be easily extensible that would be less code than the POM file for the project. But it's not so it isn't.

Many things hold us back.

The industry has been so traumzatized by slow compiles that trading speed for extensibility doesn't sell to the people who create languages. Also once you have the sophistication to make things like parser generators you know how to get things done with the terribly unergonomic parser generators we have and don't have a lot of empathy for all the programmers out there who might be using parser generators if any of them were built as if usability matters.

[patterns]

While I (think) understand the sentiment (ergonomics and practicality), it is still worth studying Lisp for a number of reasons. In particular, I would recommend to everyone to study Lisp 1.5 [1] to appreciate how an entire universe of programming languages can be bootstrapped with just a few primitives and equations.

It is true that, today, we have a vast array of impressive tools at our disposal, from parser combinators and generators, code generators to entire language workbenches with projectional editing capabilities. However, if one were to design a language for any reason, having a deep understanding of the expressiveness of computational models such as the lambda calculus would certainly be an "advantage" (especially for those who end up having to use the language): A Lisp/Scheme is as close to interactive lambda calculus as it gets. From there on, one can learn about implementing different evaluation strategies, scoping rules and continuations (William Byrd's "The Most Beautiful Program Ever Written" [2] also to mind).

Now, I am not sure whether homoiconicity tends to lead people astray. It is true that is it not strictly necessary to make a language extensible (e.g., Smalltalk has no macros and is very extensible due to its powerful meta-object protocol and an elegant syntax for closures), but it's still worth studying the concept. For example, writing a metaintpreter in Prolog [3] is surprisingly easy because of its homoiconicity.

[1] https://softwarepreservation.computerhistory.org/LISP/book/L...

[2] https://www.youtube.com/watch?v=OyfBQmvr2Hc

[3] https://www.metalevel.at/acomip/

[BoingBoomTschak]

You're really misrepresenting Lisp macros here, being able to rewrite syntax is only one (admittedly important) facet of these.

The real thing with CL is that the entire language is available during parsing and macro expansion and that users can hook into these steps to influence them. No artificial limitations like you get in C++'s consteval, you can do everything and anything without having to use a crappy DSL to do it.

Also, I've never found SBCL slow to compile. Have you?

[PaulHoule]

(1) SBCL is fine in terms of speed so far as I am concerned. But wherever you have software engineering in the large, especially involving C++, you have 40 minute builds and a split between smart people who think that's a problem and smart people who don't.

(2) The trouble with those balls-to-the-walls macros is that systems that use them tend to be "write only", like some guy writes 4000 lines of Lisp that do the work of 80,000 lines of C++ and then he moves on and there is never a version 2.0 but the system gets rewritten in C++, supposedly for performance, but really because they couldn't find anybody who could maintain it. It is really pleasing to see how little code is in Lisp systems from the golden age of AI but... when Yahoo! bought Graham's startup they kept the customers but threw away the code.

You are more likely to be sustainable if you get two people to write 12,000 lines in something like typescript or ML that uses Dragon Book technology to the same end.

[loose-cannon]

I thought Scheme was just an educational tool - a minimal language that allows the student to explore language construction & design relatively quickly. I'm not sure I understand your comment about homoiconicity leading people astray. Isn't that the characteristic feature of lisp? Doesn't that dramatically simplify generating code (in principle)?

[jrapdx3]

I find it interesting that a number of PLs were initially developed as educational or research tools but later expanded for general use. Some examples are Pascal, Scheme, Prolog. AFAIK many development languages are used in education so the distinction is rather fuzzy.

Several Scheme implementations, e.g., Chicken, Chez, Gambit, are capable of producing almost any type of program. I'm most familiar with Chicken Scheme. Comes with a large "standard" library plus a broad range of available extensions. Its easy to use FFI adds to productivity.

My experience anyway. Of course, mileage varies widely among users of any language.

[loose-cannon]

Yea sorry. I didn't mean to criticize or diminish its value by saying it's just an educational tool. It's a powerful tool that lets you do some advanced things quickly. That's totally valuable outside of an educational setting. And I didn't mean to imply that there's any limitation to its use.

I just don't understand the criticism in the post I was replying to.

[Jtsummers]

> I thought Scheme was just an educational tool

That's one use. It's also used in commercial and open source systems. Guile, for instance, has been used as an extension language (like lua and others) in many projects.

[kazinator]

> the techniques in the Dragon Book for writing compilers are the real magic.

Can confirm; got a few hints from the 1988 edition when working on the TXR Lisp compiler.

Oh, and also when implementing regexes, not to forget!

However, the Lisp compiler has its own magics that are not exactly covered in the Dragon Book. Some really nice way of doing things.

[hajile]

I might buy this if those languages were actually faster. Instead, implementations of scheme with no commercial backers run circles around scripting languages with huge budgets.

Common Lisp can match and exceed languages like Java without the backing of a megacorp.

Carp Lisp is basically identical to C in performance and old PS games were shipped in a proprietary dialect (GOAL) and had very good performance.

Lisp has proven itself capable great performance at every abstraction level.

[pjmlp]

Except that CPython still misses on (compile .....) part, indeed holding the industry back.

[guenthert]

Not sure about that. I rather think it's the very dynamic nature of Python (e.g. at run time one can add members to classes) which limits performance gains achievable by compilation. Iirc PyPy got to some five fold performance gain against CPython.

Common Lisp, as an example, makes some deliberate flexibility (and elegance) vs. performance trade-offs. I'd think to speed Python up more than PyPy could, one would have to sacrifice some of its features (some of those might not be all that popular or even desirable).

Nah, Python has its place. If its performance isn't adequate for the problem at hand, rewrite the bottlenecks in C++ and be done. LLMs should make that easy, no?

[pjmlp]

The dynamic excuse always misses out fully dynamic languages like Smalltalk, Self, among others have achieved, and how it got further exploited into JavaScript engines.

However CPython isn't even half way there to either Papyrus or GraalPy.

The community attitude to rewrite into C, C++, Rust, Zig, whatever, is exactly why Python will never be like other dynamic languages, and the dynamism excuse will always be used.

Thankfully at least GPU vendors see it otherwise, so if nothing else at least it will get JITs that speak GPU machine code.

[BoingBoomTschak]

> e.g. at run time one can add members to classes

Which is something that can be done in CLOS, though. And in a much more thought out fashion, as usual with the CLOS/MOP, cf https://www.lispworks.com/documentation/HyperSpec/Body/04_cf...

[PaulHoule]

it compiles to bytecode but the bytecode engine is not that fast.

i had a phase when I was using PyPi a lot for branchy "old AI" kinds of workloads and felt it was an easy win but since then it has been either numpy or PIL or pytorch doing the heavy lifting or scripty stuff like uploading files to S3 where performance doesn't matter a lot.

I will grant that Common Lisp can be compiled to run amazingly quickly!

[BoingBoomTschak]

What he probably meant is that compilation is available at runtime to the user, allowing you to JIT stuff at will.

This is how https://github.com/marcoheisig/Petalisp#why-is-petalisp-writ... and https://github.com/numcl/specialized-function can exist.

[pjmlp]

In Lisp (compile ...) produces machine code, with various optimisation flags available, depending on the implementation.

[iLemming]

There's nothing "special" about Lisp and Lisp dialects, yes. Similar features can be or already have been implemented in other languages. Yet, after touching, using and experiencing working with a bunch of different stacks, I cannot simply ignore the enormous pragmatic level of Lisps.

Working with Clojure is an absolute delight. It strips down all the dogma and let's you deal with the "business logic" as if you're cooking steak using no BS ingredients - meat is meat, herbs are real, stove is hot.

Why would I ever choose bash for writing anything slightly more complex than simple redirection, when I can do things in way better fashion with babashka. Why would I wrestle a YAML CI pipeline that only fails on push, when I can drive the whole thing from a babashka task file, run each step locally in the REPL, and actually debug it?

Why would I ever deal with Lua, if I can't even format it for "readability" - no matter how I do it, it just looks darn ugly, and luafmt often makes it worse. Why, if I can just slash down dozen lines of Lua boilerplate compressing it into a three-liner Fennel macro? With Fennel, I can interactively poke through elements of my WM through Hammerspoon on Mac, and that's just bananas.

Why would I ever deal with JSON, when EDN is almost twice as compact and far more readable - I can align things and treat data as a literal table. Besides, I can group, sort, filter, slice, dice, salt & pepper that data easily, without ever leaving my trusted editor.

Why would I choose to build a web-scraper in Python, when I can use nbb driving Playwright and go through selectors interactively, directly from my editor, as if it is a devtools console. And I don't even have to restart anything, deal with state changes, etc.

How can I abandon Emacs where I can just open a scratch buffer, type some Elisp and change the behavior of my editor, my WM, my OS and even things on remote computers. No other text editing environment works the way Emacs does - nothing even comes close. It feels like playing a video game, where my controller in my editor.

Why would I write Flutter UIs in Dart, fighting the widget-tree ceremony and endless build() boilerplate, when ClojureDart lets me express the same tree as plain data and hot-reload it interactively? The layout is just nested maps and vectors.

Why would I reach for C when I need to embed a small, fast scripting layer. Text parsing alone would be a regex nightmare elsewhere.

Why would I bolt a templating engine onto HTML strings, when Hiccup makes markup just vectors - so my views compose, filter, and generate like any other data, no special templating DSL to learn

And with all sorts of different runtimes and dissimilar Lisp dialects, it still feels as if you're working with the same language. The mental overhead when switching is so negligible. While switching between just JS and TS - which are supposed to be of the "same family" - feels quite annoying. Despite the fact that I've put years into those - far longer than any Lisp I've ever used.

Sure, nothing special about Lisp at all. Except that practicing Lisp can actually make you a polyglot. You'd realize that it isn't syntax that makes a programming language, but runtime and semantics do. After years of dealing with different PLs, I lost a preference for one specific language - I'd choose the runtime best suitable for the task, and then see if I can bolt Lisp on top of it. And these days, it feels like there isn't a platform left where you can't meaningfully do things via Lisp.

[kazinator]

1. Languages that have all the Lisp features are in the Lisp family. Thus languages that are not in the Lisp familly are ones that don't have all the Lisp features. Once a langauge disappears behind the event horizon of the Lisp family, outsiders no longer distinguish it from any other member of the Lisp family; it's just Lisp.

2. Having some features of Lisp is not the same thing as having those features in that form and integrated in that way.

It's not just the roster of features, but how they blend!

If two features have to speak with each other through some interpretation layer, that spoils everything.

[PaulHoule]

I've got really mixed feelings about Clojure. Like when I read Graham's book I went through a phase of "he wouldn't be fighting with nconc if he was using Clojure!" I find some algorithms to be easy and fun to write using persistent data structures and others maddening though if I did it more I'd get better at it. zippers are weirdly unergonomic because of some little bad choices. I badly wanted to do a project with the Clara rules engine but never got the chance.

As for GNU emacs I got over it. Like I was introduced to it in 1989, it was the cool new editor for the Unix world, one of my friends loved scripting with it. However I experienced it getting really janky about 20 years before other software got janky and I don't know why. Some of it might have been rot in things it depends on, like it used to be curses apps actually worked right but now they suffer data corruption from the telnet/ssh connection and always seem to have an off-by-one on the screen size. X Windows got glitchier. I guess emacs was ahead of its time, now people write janky apps with Electron -- like I have to type R E A L L Y : S L O W L Y when I use Slack at work. Photoshop is like that too, computers are 100x's of times faster so Photoshop should boot up faster than Momo Chiyoda can transform but no, it doesn't.

I switched to vim around 2003 as my emergency Unix editor because I could log into a busted system and start working and not have to rely on the package system (often the thing that was busted) to install emacs or build it from scratch. Helps that it doesn't use continuation characters so cut and past just works, there are lots of little things that make vim live more comfortably side by side with modern GUI life than emacs does.

[iLemming]

> when I use Slack at work

Years ago I stopped typing anything longer than three words in anything but my editor, I just don't even understand why every programmer doesn't do that - it just makes perfect sense. The idea of automating copy+paste from any program into your editor and back is such an obvious thing, it feels stupid if you're not doing that.

> there are lots of little things that make vim live more comfortably side by side with modern GUI life than emacs does.

No disrespect, but honestly it sounds like you have never experienced the practical side of a Lisp system that permeates your OS. You just can't meaningfully compare Vim (or pretty much any other text editor) and Emacs, because Emacs is not a text editor, it's rather a text orchestrator. I for example easily build such workflows like "grab the url from one of my browser tabs and insert it", or I can get it from my browser history, or "retrive current thread I see in Slack and search for specific piece of text". Or placing the cursor to the plain text like "FOO-4142" immediately gets pattern matched and I see the popup with the Jira ticket description. That all I do in the middle of typing any text, like for example this very comment. I don't have to leave my editor, I'm not forced to do things how they dictated by other apps.

I use Neovim almost daily, it comes in handy for quick text editing in the terminal, and I love the idea of Vim in general - it's a beautiful, practical, amazing model and I use it everywhere, it permeates my OS - my WM, all my editors, my terminal and my browsers. But Neovim just can't be like Emacs, because fundamentally they are categorically incomparable. In practice that means that the "comfortably side by side" reach you're trying to describe goes further, because of affordances - building specific workflows that affect things on your local and remote machines is cheaper, quicker and simpler. I would grab any area of my screen and it gets immediately OCRed and extracted text pops in an Emacs buffer. It took me fifteen minutes to build that workflow. Getting done anything similar in pretty much any other editor wouldn't even occur to me - I wouldn't even bother seeing it as a problem that needs solving. Affordances shape the way you do things: https://news.ycombinator.com/item?id=48876315

[spopejoy]

I'm a longtime emacs head and partisan ... but I learned vi to compose emails in the Berkeley computer lab in 1989 and somehow have never forgotten vi key commands ever since. Emacs is my preferred sw dev editor for most things but I feel like not knowing vi is almost professional negligence if you are on the terminal.

[iLemming]

I know, right? When people respond to a sentence containing "vim", with "meh", I feel like going: "What do you mean 'meh'? Are you saying you never used sed, or haven't heard of manpages, or `less` and `more`? What kind of programming you've been doing so far?"

How the heck anyone doing anything remotely related to programming can't be familiar with vim navigation? And if they're familiar what does "meh" supposed to mean? It's like having an "opinion" about where the Tab key sits on the keyboard: "I hate Tab, it was a stupid idea to put it on the left side..."