[wizzledonker]

I guess, just to play devils advocate, what if code turns out to just be the minimally expressive communication medium required for a human to maintain agency over the function of a system?

If we consider (and I’m not saying this is true, just for arguments sake) that this agency is important as other humans enjoy talking to humans about a goal, rather than machines, and we think this will never change.

In this peculiar case, it actually doesn’t matter how “good” llms get - they will never cross a barrier fundamental to human nature?

[ilaksh]

Maybe you're trying to argue that LLMs won't be able to simulate humans or reproduce all of human nature. That's true, but it's not what I was discussing. I was talking about programming.

Also, they are really VLMs and VLAs and there are other types of architecture and there will be many more.

So if you think that there is something salient about humans that can't be reproduced eventually by AI, then look into religious dualism.

[wizzledonker]

> So if you think that there is something salient about humans that can't be reproduced eventually by AI, then look into religious dualism.

Contradicts

> not what I was discussing. I was talking about programming.

To be concrete: my argument is that to

> "simulate humans or reproduce all of human nature"

Is exactly what would be required for an LLM to replace a human as an effective interface for programming. Crazy hill to die on, I know, but that's the goal.

I am purely talking about programming. The reasoning looks something like this.

1. LLMs are undeniably useful as assistants when programming.

2. At some point though, they are leveraged too much. This leads to humans having an inability to reason about the generated code.

3. The LLM now needs to be as good of an interface as a human being to be of any use.

I think most people would agree with this distinction. Where people argue is where the cutoff point lies - and it's actually hard to find. How do you know when you have transitioned between the three stages? It's tempting to say, if the speedup is marginal, it isn't worth the risk.