[Hammershaft]

It's fascinating that Clojure has consistently the best performing solutions and yet at the same time such a low success rate.

Do you have an idea as to why that is?

If I had to guess, two things lowering reliability:

A) Balancing parens might be tough on an LLM one-shot.

B) LLMs generate tokens sequentially, but s-expressions mean the first forms to be evaluated in a body are usually the last to be written, so the LLM has to sequentially generate layers of evaluation backwards.

[jwr]

First, we would need to agree on what "such a low success rate" means. Programmers have a thundering herd mentality: there are usually 2-3 "top things" that are in fashion at any given time and the herd tends to go towards these top things. They are not necessarily good or "successful" (however you define that term), they are just popular today.

From my point of view, Clojure is a very successful language. It has been in stable development for >10 years now, with no major breaking changes (!). I was able to start a business using it and now make a living from it, all of it possible largely because Clojure reduces incidental complexity so much.

Now, as to LLMs, I can see this discussion is mostly theoretical, so let me pitch in with data. I've been using LLMs for Clojure for a while now and it works fantastically, from what I read about other languages, quite a bit better for me than for others. Balancing parens was a problem for early LLMs without tools, Claude Opus with clojure-mcp tools doesn't encounter that problem at all.

Additionally, the ability to try things in the REPL means that LLMs are very effective: all hypotheses and solutions are immediately tested, with automatic feedback.

Overall I get great value from LLMs and I am able to solve large problems with them.

[regularfry]

I've found it helps to give the model a lower nesting limit than you might give a human who has access to a paren-balancing editor. If all functions are shallow, there's less opportunity for paren balancing to get out of control, and reasoning about the evaluation flow doesn't have to jump back and forth so much.

This also doesn't hurt the code from a human reader's point of view.

[xoxolian]

If B), then maybe the LLM should be instructed to prefer things like the -> and ->> operators. So the first forms evaluated are also the first written.