[tptacek]

People keep saying this and it doesn't make sense. I review code. I don't construct a theory of mind of the author of the code. With AI-generated code, if it isn't eminently reviewable, I reflexively kill the PR and either try again or change the tasking.

There's always this vibe that, like, AI code is like an IOCCC puzzle. No. It's extremely boring mid-code. Any competent developer can review it.

[daveguy]

You construct a theory of mind of the author of a work whether you recognize you are doing it or not. There are certain things everyone assumes about code based on the fact that we expect someone who writes code to have simple common sense. Which, of course, LLMs do not.

When you are talking to a person and interpreting what they mean, you have an inherent theory of mind whether you are consciously thinking "how does this person think" or not. It's how we communicate with other people efficiently and it's one of the many things missing with LLM roulette. It's not that you generate a new "theory of mind" with every interaction. It's not something you have to consciously do (although you can, like breathing).

[buu700]

I assumed they were describing AI itself as a black box (contrasting it with deterministic code generation), not the output of AI.

[tptacek]

Right, I get that, and an LLM call by itself clearly is a black box. I just don't get why that's supposed to matter. It produces an artifact I can (and must) verify myself.

[buu700]

Because if the LLM is a black box and its output must ultimately be verified by humans, then you can't treat conversion of prompts into code as a simple build step as though an AI agent were just some sort of compiler. You still need to persist the actual code in source control.

(I assume that isn't what you're actually arguing against, in which case at least one of us must have misread something from the parent chain.)

[tptacek]

Right, you definitely can't do that. People do talk as if the question was whether we could stick LLM calls into Makefiles. Nobody would ever do that, at least not with the technology we have at hand.

[layer8]

But that’s exactly what the part of the article quoted in the root comment (https://news.ycombinator.com/item?id=44206758) is about, and in reference to future AI technology. That’s was this subthread was discussing.

[tptacek]

Yep, you're 100% right. Sorry!

[baq]

Ever is a long time. I expect first products built this exact way working reliably and having happy customers in the next five years, pessimistically. Optimistically this is probably happening somewhere as we speak.