[Zak]

> it’s hard to imagine they’ll ever independently develop a compiler for a truly novel programming language

I did exactly that using an LLM. It may not count as independent depending on how strict you are about that, but then LLMs don't do anything independently.

I wrote a small sample program and expected output, then told the LLM to write a compiler for it in C using LLVM. I subsequently told it to extend the language until it could be used for its own compiler, and rewrite the compiler in the new language. It did.

I don't think that contradicts your point about creativity. A compiler is probably a more mechanical task than a CRUD app is. There's a non-negotiable definition of done and correct.

Designing a language is a creative task of course, and I wouldn't expect an LLM to come up with a novel or ergonomic design on its own. In fact subsequent experiments have shown me that LLMs will consistently ignore terrible ergonomics in a language, never seeking opportunities to add abstraction or beauty.

[fn-mote]

My main argument about this kind of work is that it is (essentially if not exactly) in the training set.

I want a different argument before I believe that LLMs are doing “out of training dataspace creativity” (extrapolation not interpolation).

[versteegen]

It's true. Creative work is in the training set.

Being serious. "Think about what this might mean..." Finding unexpected links between various ideas and background knowledge. What we call a "novel idea" is virtually always the repurposing of an idea/concept in a new context. A system that maps arbitrary inputs into abstraction spaces in which similarities are discoverable, such as say a deep learning system, is perfect for this.

[cedws]

But what was novel about the language?

[Zak]

At the time the LLM generated the compiler, just syntax. The semantics it generated were the subset of C that the C version of the compiler already used.

You could set the goalposts such that it wasn't novel enough to count, but for a short time I had code running in a language that nobody had ever known. Getting it from that point to a language that's ergonomic to use, teaches something about computing, or both is a longer journey, and certainly not one an LLM could take on its own.

An LLM won't come up with an interesting CRUD app on its own either. Parts of that process are pretty mechanical, but we had skeletons and templates before we had LLMs.