[goranmoomin]

I like Lisp, I’ve used Common Lisp with a passion, but this doesn’t seem like a valid argument for Lisp.

Homoiconicity, as I understand, is that the code is structured data that is easy to programmatically modify, hence allowing Lisp macros. While some might disagree, I see Rust macros as the closest thing that demonstrates homoiconicity in mainstream Algol-based languages, as Rust macros modify the loosely structured token stream to produce new Rust code.

Eval, on the other hand, that’s more of a capability that comes from Lisp’s runtime, which used to be unique when Lisp was thriving, but not anymore — JS, Python, Ruby, all of the runtime-based languages have an eval function. The fact that they are not used as much is more of a security issue, not a capability issue, and I am not sure how having eval can be argued as Lisp being the language of agents.

[miki123211]

> I see Rust macros as the closest thing that demonstrates homoiconicity in mainstream Algol-based languages

I don't know if you consider Elixir mainstream, but IMO their macro system is much closer to lisp's ideal.

Elixir is basically Lisp, but with better syntax, a modern ecosystem, and running on the Beam. Unlike languages like Rust, Elixir's conditionals and function definitions are just calls in the AST, even though the syntax looks mainstream and not like paren soup.

[embedding-shape]

> Elixir is basically Lisp, but with better syntax

Which so helpfully removes many of the benefits of lisps :P I don't understand this argument at all, if it's not s-expressions nor quacks like a lisp, in what way is it lisp at all? Making it algol/C-like already makes it like the rest of the 99% of the languages, without any of the easy benefits of the neat and simple syntax of lisps.

[drob518]

Agreed. You can tell when someone “gets it” or not by whether they see sexprs as a problem to be overcome or a benefit to be embraced. Count me firmly in the benefit camp.

[galaxyLogic]

A Lisp program that writes a Lisp program really just needs to produce a list of (nested lists) of tokens. A JavaScript program that writes a javascript program needs to generate a string that is syntactically valid JavaScript code. That is a much bigger task than just constructing a (nested) list.

Because Lisp syntax is so much simpler than that of JavaScript etc. it is much easier to avoid errors when generating code. In JavaScript you can use JSON to generate data, but JSON can not carry functions around.

I think this idea makes a lot sense. Instead of making the LLM generate JSON or XML, why not make it generate Lisp, which can carry both programs and data?

[gf000]

Well, then ad absurdum it would be easier to just output a bunch of 0s and 1s, right?

The same goes for "just a list of nested lists", sure it is easy to produce it, and for trivial examples it may actually be easy, but for more complex and realistic problems lack of "higher order" structure is a negative!

For something like JavaScript, you can just have a language-native AST object with a few helper functions and then can just call "addFunction" on it with proper arguments so that the API shape validates plenty of properties of your output.

[ijk]

Except you can express the higher order structure in s-expressions too, and using an AST usually means expressing it as a different syntax than the other parts of your program.

Or maybe I've just been sleeping on the power of Javascript AST.

[gf000]

So can you in binary!

And that's more of a library ergonomics question how nice it is. There are plenty of other languages with nice ASTs and `eval`, and that's the only ingredient you need.

[zingar]

I haven’t seen an LLM generate syntactically invalid code in any language in … a year? So I don’t know if “more likely to be syntactically valid” is a good enough selling point.

But maybe I’m interpreting you too narrowly?

[Zambyte]

I take it you never run models locally then? Local models that can run on reasonable consumer hardware have gotten to the point of being very useful, but they still occasionally are tripped up with syntax errors. Especially if you quantize the model and KV.

[zingar]

Ah of course, I do have a blind spot with local models.

[galaxyLogic]

I'm thinking that there should be some standard for Agent-LLM communications, and the simpler the better. Lisp might be such a simpler standard as opposed to JSON.

[goranmoomin]

> A Lisp program that writes a Lisp program really just needs to produce a list of (nested lists) of tokens. A JavaScript program that writes a javascript program needs to generate a string that is syntactically valid JavaScript code. That is a much bigger task than just constructing a (nested) list.

> Because Lisp syntax is so much simpler than that of JavaScript etc. it is much easier to avoid errors when generating code. In JavaScript you can use JSON to generate data, but JSON can not carry functions around.

First of all, the LLM does not produce structured tokens, it produces (tokens of) text. It does not have a concept of nested or structured tokens. Which means that producing a Lisp program and a JavaScript program is basically the same difficulty, i.e. LLMs producing function foo () {} is about the same task as producing (defun foo () ()).

In fact, because most Lisps uses the same token ( and ) for almost all delimiters, the LLM in fact gets confused a lot more than Algol-family languages that uses various different tokens for different purposes. It generates thinking traces that are a few screens long while trying to count the closing parenthesis and the depth, something that I have not found to be the case for other languages, even with languages much more obscure than Lisp. (And no, it is not a training data issue, because the Lisp family as a whole is pretty well represented in the data set, due to Emacs Lisp.)

> I think this idea makes a lot sense. Instead of making the LLM generate JSON or XML, why not make it generate Lisp, which can carry both programs and data?

You do realize that all programming languages contains both programs and data, right? i.e. JSON is literally a subset of the JavaScript language, all JSON documents are valid JavaScript code, can be embedded in JavaScript programs, and so on. This isn't even a JavaScript-specific thing, almost all recent programming languages have data structure literals.

The thing that makes Lisp unique is that it can modify programs as data in the language easily, and why would that be a unique capability that would be beneficial for LLMs, when it can just sed/awk or tree-sitter-parse programs with more conventional languages and modify it as easily?

[zingar]

Now that you mention it, Claude has had trouble at times with balancing brackets in the elisp I have it write.

Yet I still had the idea that LLMs should be better at lisp than other languages.

A fascinating contradiction, thanks for pointing this out.

[ijk]

Yeah, unfortunately "LLMs are bad at counting" seems to apply to counting parentheses.

With modern tool calling I wonder if a better way to go about it is for the LLM to express the program changes as a function or otherwise use an editor that auto-balances parens. There's a lot of relatively simple tooling that makes it easier to write in a Lisp. The languages tend to lend themselves to being straightforward to check like that.

What's special about Lisp's repl is that it's perfectly possible to construct your entire program in the repl, testing each addition live as you write it. (Many Lisp-focused editors assume you'll want to do this, such as Emacs making it easy to run the interpreter on a single function in a file.) That tooling is lost if you just try to one-shot the file, and before 2026 the majority of LLMs originally just tried to one-shot every file.

But, just like a lot of early LLMs had huge problems with whitespace and numbers because the tokenization was taking efficiency shortcuts that made sense for text but absolutely wrecked code syntaxe, I wonder if the current optimizations are badly formatted for Lisp.

At the very least, using a varient like Clojure that also uses [] and {} in addition to () might help.

[cma]

This deepmind paper showed big limitations on nested structures with transformers, LSTMs actually did better:

Neural Networks and the Chomsky Hierarchy

https://arxiv.org/abs/2207.02098

[galaxyLogic]

Interesting. Can someone quickly summarize their results? Are they saying that using languages with simpler syntax Lisp lead to LLM being able to generalize their knowledge better?

[cma]

They tested them on formal languages of different power, and saw where they could generalize beyond the training data. Transformers failed to generalize pretty early on at stack machines/brace matching.

It was a good bit older of a paper though, if I remember it's somewhat expected from the pure feed forward nature of them and limited circuit depth, where LSTMs have some recurrence.

[Zambyte]

I believe the problem stems from ) and )) both being one token wide. I wonder if repeated parentheses being removed from the token vocabulary would improve accuracy for Lisp code generation.

[galaxyLogic]

> JSON is literally a subset of the JavaScript language

Isn't it more like when you write JSON in JavaScript you write it as a String? JavaScript has the data-type "String" but does it also have a built-in data-type "JSON"?

You can pass such a string containing JSON to (built-in) library function JSON.parse() and you can take a data-structure written in JavaScript can call JSON.stringify() on it. But if your JSON is malformed, does the JS-interpreter tell you that, before you call the library-function JUSON.parse() ?

If you have a JavaScript parser/interpreter, does it parse JSON too? If not I would say JSON is not part of the JavaScript language, even if JS runtime provides a library function for parsing it.

[TacticalCoder]

> Eval, on the other hand, that’s more of a capability that comes from Lisp’s runtime, which used to be unique when Lisp was thriving, but not anymore — JS, Python, Ruby, all of the runtime-based languages have an eval function.

And it's good we have those for troubleshooting but those eval still offer nowhere near the power of a Lisp REPL.

> and I am not sure how having eval can be argued as Lisp being the language of agents

I've seen several Lisp programmers saying that it's really the REPL (and the 'E' in REPL is for "Eval") that is the godsend when working with LLMs.

With LLMs we've seen terminals/ssh/tmux and CLI tools calling making a huge comeback (not that they ever went very far).

Now I wouldn't be surprised if we were soon to see a Lisp AI harness also using extensively the REPL becoming succesful.

It's too early to tell it's not a powerful combo (LLMs + Lisp REPL).

[gf000]

How is that different from other languages that have proper repls? Lisps don't really give you much here, if the language can parse itself and can eval it, you have the sufficient primitives.

What helps is how "small" the hot reloaded parts can get - lisps are good here due to a lot of functions and not much shared state. But that is again, not something specific to lisps, neither are they the best at it - there are far purer FP languages for example

[mvc]

Does eval in these other languages evaluate ASTs or strings? It makes a difference.

[gf000]

eval(parse(yourString))

No it doesn't

[Shorel]

It's not just eval. It's LISP macros what makes it an interesting powerful language for agents.

[ignoreusernames]

> Homoiconicity, as I understand, is that the code is structured data that is easy to programmatically modify

I think a more accurate description is that lisp code is just cons cells and cons cells is both how we write the code and how the runtime itself implements lists. So there’s basically no distinction between a lisp list and the text representation of the source. Rust and its macros is a different situation because the text representation of the code and the syntax tree have completely different shapes

[embedding-shape]

> I think a more accurate description is that lisp code is just cons cells and cons cells is both how we write the code and how the runtime itself implements lists. So there’s basically no distinction between a lisp list and the text representation of the source.

Heresy I suppose, but doesn't feel (to me) like it "specifically" has to be cons cells, as long as it's a "list" of some sort, regardless of the specific type of list, while I'd say "cons cells" is a specific implementation of a list, famously implemented by Common Lisp and similar lisps.

Besides that I agree with you, the code you write and the code the compiler uses is the same, that's why it gets easy to edit with code itself. Rust and similar don't have this feature at all, it's very different experience writing macros with lots of special syntax, compared to just writing "normal" code doing the same sort of operations. "Doing a loop of X" isn't the same inside of a normal runtime function body, as when you want to do a loop of arguments for the function signature in a macro, as just one simple example.

Not to mention in Rust you have two different types of macros, declarative vs procedural macros, already speaking to that the entire concept of homoiconicity isn't there. You want it to be easy to implement a read, evaluate, print and loop back workflow by passing native data through the entire chain, except parsing the initial user-input string into your AST (whatever shape that ends up being), which pretty-printed (basically) looks the same as the user-input.

[ignoreusernames]

> Heresy I suppose, but doesn't feel (to me) like it "specifically" has to be cons cells, as long as it's a "list" of some sort

Yeah, "cons" is definitely a implementation detail. Maybe the central ideia is just "parenthesized s-expr".

[gf000]

But this is just not true. An utf8 string is most definitely not a bunch of cons cells. Having a one-to-one correspondence is not the same thing, and one might argue that a rust code snippet and the corresponding AST object is just as arbitrarily far apart with a clean one-to-one correspondence.

So like, what's the material difference between the two? You have a parse and an eval in both cases