[ulrikrasmussen]

Can't remember who said it, but I think the most poignant commentary about the prospects of LLMs completely replacing humans is the simple observation that yes, you can delegate many if not all of your tasks requiring thinking to LLMs, but it is inherently impossible to delegate the task of understanding. If you hope to write an efficient prompt which gives you what you think you want, you need to actually understand what you want, which also requires you to understand what you currently have. Programming languages are a medium for transferring and - not the least - maintaining shared understanding of a software system in terms of the basic abstractions that the programming language provides. The need for good programming languages does not go away just because we use LLMs to write the code, on the contrary: would you rather review LLM-written assembly or LLM-written Haskell?

[willtemperley]

This is absolutely the headache I’m having today, an enthusiastic contributor pasting AI generated wall-of-text comments to justify their 500 line PR.

How can we have a conversation and reach a shared understanding if one of the participants is avoiding attempting to understand the problem?

I have no problem with AI generated code, provided it’s reviewed, but I might as well be talking to a brick wall if the person hasn’t understood.

[lelanthran]

> his is absolutely the headache I’m having today, an enthusiastic contributor pasting AI generated wall-of-text comments to justify their 500 line PR.

Easily solved - call that contributor into a meeting to explain their PR. If they cannot explain it verbally, it would be made clear to them why it cannot be approved without you having to actually say anything.

In the good old days (SVN), code reviews at places I worked at were done by booking one of the many small rooms and throwing the diff up on a projector. The author could defend, explain, etc.

Maybe we'll have to return to that type of code review from now on.

[xscott]

> would you rather review LLM-written assembly or LLM-written Haskell?

I wish we had a language that was targeted specifically for LLMs to write and humans and LLMs to inspect:

- Simple robust syntax

- One obvious way to do things

- Static type checking

- Purely functional encouraged, escape hatches for performance

- Inspect-able, testable, and reviewable in small pieces

- Something like formal predicates, preconditions, post-conditions, assertions, or effects typing

Giving LLMs all the surface area of Python, JavaScript, TypeScript, or C++ seems like a huge mistake. It's amazing it works as well as it does. Well written Haskell is beautiful, but there are way too many ways to write Haskell:

https://people.willamette.edu/~fruehr/haskell/evolution.html

[ulrikrasmussen]

I don't really understand why we need an LLM-specific programming language - all of the properties you list are properties that modern programming language designers already try to achieve.

> - Inspect-able, testable, and reviewable in small pieces

This is more a property of the architecture than a property of the language, although some programming languages support it better than others.

[B-Con]

Go seems like a pragmatic fit, even though it was obviously not designed for LLMs and nor is it purely functional.

I review a lot of Go code, which means I review a lot of LLM Go code. Even though the human vs LLM authorship distinction has strong signals, Go's simple nature seems like a useful constraint on how LLMs can express themselves.

[jbreckmckye]

- Too verbose, too much noise e.g. error handling.

- Type system is too concrete, can't express sets or unions.

- Not much support for functional programming except passing closures

- Can't express immutability. Well, there's const, but it's crippled

These things mean Go falls short of what GP wants.

But Go has very distinct and (honestly kinda weird) design goals, it isn't really supposed to be "the simple applications language". It's supposed to be "C with NewSqueak", very much a systems programming language. You see that in its aggressively concrete type system

[actionfromafar]

I agree it's easy to read. But you also have to read a lot of it, before anything interesting happens.

[NortySpock]

Gleam ?

Elixir's exhaustive pattern matching and gradual type checking is also on my radar.

I guess I don't know what you mean by effects typing.

[lelanthran]

> I wish we had a language that was targeted specifically for LLMs to write and humans and LLMs to inspect:

How would this help? LLMs are fine with existing languages' syntax, semantics, idioms, etc. Where they fall over is in understanding intent of a prompt within the context of architecture, systems, roadmap, etc.

[IsTom]

> escape hatches for performance

These allow the model to cheat in ways that are harder to detect. In actually pure FP you can be sure that it's not counting function invocations to bypass tests.

[xscott]

Yeah, but sorting algs, FFTs, matrix factorizations, backprop, and many other things just aren't the same with purely functional data structures.

Maybe these well known cases could be hidden in the API, but it's easy to come up with other examples. I forget what Clojure calls it, but they have some notion about things where they're mutable during "birth" and then locked down.

Someone smarter than me knows how to do this right.

[IsTom]

That sounds like Haskell's ST monad and yeah, that could work. These are very specific "escape hatches" though and typically people want more.

[xscott]

As far as I can tell, the good models will use whatever you give them. So it seems we should only give them language features that help humans understand and maintain what the model writes.

Amusingly, a friend had Claude write some BrainF*ck the other day. Non-trivial algorithm, and it made working code on the first try.

[sriku]

Lean seems to fit your ask fairly well.

[empty_set35204]

I think ocaml is fairly close to this to be honest.

[avadodin]

I always find it amazing that they can produce syntactically valid Python code —which is basically Whitespace with some syntactic sugar— most of the time while being utterly unable to count the rs in strawberry.

[reichstein]

LLMs to inspect:

- Simple robust syntax

> - One obvious way to do things

That always leads to more verbosity. Much syntactic sugar is a more specialized way to do a subset of a more general thing. Why have `a + b` when you could just write `a.add(b)`? Because it's easier to read. The general functionality needs to account for all cases, the specialized one can cut that down just the things that matter to a specific common use case.

- Static type checking

\<meme>Which one?\</meme>

That's a very, very deep can of worms. One could argue that of an LLM is generating the code, then the type system doesn't need to be understandable to humans, so throw in all the features you'd ever want and just have the LLM change the code if it's not valid. Unions of higher order generic functions, sure! Or one could argue that there should be minimal magic, because the LLM understands the language only by its source, so everything should be explicit. If the LLM can prove that something is sounds to the compiler, accept it. Give ways to give extra evidence of soundness, like declaring invariants and contacts.

- Purely functional encouraged, escape hatches for performance

"One obvious way to do things", except when you need two. Purely functional except when it matters.

Why doesn't performance always matter? (And how will an LLM know if it does?)

Being purely functional is nice for data, but not for data structures that are updated in place. If all you do is stream data from one DB query into another, then your mutable state is the database. Otherwise might as well accept that it's a multiparadigmatic language with both imperative, functional and OO features. Just like all the others.

- Inspect-able, testable, and reviewable in small pieces

Good modularity and abstraction. No global scope. Maybe something like dependency injection to decouple from dependencies? (That does not make code readable, though.)

- Something like formal predicates, preconditions, post-conditions, assertions, or effects typing

That! LLMs look at the source. The more explicit the source is, the less it has to infer from context or existing knowledge. If the LLM can create its own predicates, accepted by the static type/analysis system, to prove that it's code is sounds, that allows more flexibility than having to fit into any fixed type system. Do we want or need that flexibility? Maybe. Most LLM-generated code is directly inspired by existing idiomatic code in the same language. Some is translated from other languages, trying to match up idioms. Some is just blindly trying to make unit tests pass. We could end up with generated predicates tailored to specific unit tests, not the actual concept.

[byzantinegene]

we are already at the point where AI proponents are planning to have the review cycle be replaced by llms

[hcfman]

That’s a little black and white though. It’s ruling out that the AI could make a pretty good guess as to what you likely wanted to convey. If it gets that right, then you didn’t need to understand it nearly much.

When I observe its reasoning I see it doing this all the time.

[ulrikrasmussen]

Sure, but at that point it hardly makes sense to discuss any of the details of the implementation since we are just treating the LLM as a black box. For all we know, it could be maintaining its own specialized programming language made just for the application.

[geraneum]

How does the AI guess?

[antonvs]

Using statistics. They’re probably better at it than we are.

[geraneum]

Stata is better in statistics than we are. Why doesn’t that produce guesses?

[Nekorosu]

The original quote is from Andrej Karpathy as far as I remember. I do like a lot of his ideas but try to take them with a pinch of salt as he clearly has a technocratic bias regarding LLMs/AI.

[simianwords]

That was Yacine Twitter account and it’s false. You can delegate the task of understanding. This looks like a popular folk theory that also falls under scrutiny.

[ulrikrasmussen]

Unless you have the technology from The Matrix which can inject knowledge into your brain, then no, you definitely cannot delegate understanding.

You can delegate the responsibility of understanding to an LLM, but that is not the same.

[jagadaga]

Do you understand every line of code written by everyone else on your team? I doubt it. But if you ever need to work with their code, that's when you might need to understand it. So, treat an AI agent like another teammate.