[jknoepfler]

This is a very bad comparison. You can see that immediately if you think to yourself "can I see a future in which we compile code with LLMs?". The answer is no. That's a terrible idea, and the fact that it's a terrible idea is obvious.

The formal foundations of compilers are completely different from the formal foundations of LLMs.

The former are deterministic, easy to formally verify, and extremely simple in nature. "Translate a for-loop into x86 instructions using a set of rules."

The latter is intrinsically statistical in nature. "Translate a human language prompt into functional code" has to infer the correct output statistically from similar, observed input->output relationships. There is no guarantee of consistency. Different builds of the model will see different input->output evidence, in different order, and parameter tuning will further change how it responds to those pieces of evidence. Evidence is incomplete. Local minima are inevitable. LLMs are lossy curve-fitters under the hood. Errors aren't an option, they're an inevitability.