[agentultra]

I always have to ask people, correct with respect to what?

The reason being that most software developers aren’t very good at specifying systems precisely enough that they can confidently determine whether their program does what they think it does.

We don’t need to specify systems to the rigour of mathematical proofs every time, of course. The point is that spoken language isn’t precise enough to measure correctness against.

When you suggest that the difference in correctness is huge, I can’t really understand what you mean. Correct with respect to how you think good software should be written? Correct with respect to informal specifications written in prose? How?

I know that the frontier models have large contexts and can compress a lot more tokens from their training sets. But how does that make it able to read your mind and make formal what is ambiguous or informal?