| It's kind of a useful distinction, but the categories are really blurry. For example, the layout software vendors provide a solution that both uses non-determinism in their final product, and also checks the results against whatever criteria you give it. If it tells you it will work, it's probably not lying, and if it tells you it won't work, it's almost certainly not lying. So in your own creative process of building a board, you can rely on that. Many LLMs don't have this property at all, and honestly, it really doesn't matter whether you are using the LLM to craft a legal document or to craft code; if you don't check the output yourself, you're possibly in for a world of hurt. > Now if you are "running" your "generated code" that's where people will have strong opinions because it conflates the two completely different processes in the worst way possible. I think a primary driver of the attractive nuisance here is that, because the execution of the resultant code is itself a deterministic process (even if the process that produced that code it isn't) people think that they can tame the process, by some combination of automated testing and looped automated reprompting. At some level, they may not be wrong -- computer chips themselves are built on top of stochastic atomic processes, and google famously proved in its early days that it could have reliable processes on top of shitty commodity hardware. But one key difference is that the nondeterminism in the atomic processes or in the non-ECC memory of the white box computers is uncorrelated. And another key difference is that, unlike the layout software vendors, so far the LLM vendors seem unable or unwilling to properly self-check their own outputs. To me, this is kind of the canary in the coal mine. You would think that the LLM vendors have every incentive to weed out bad results, that they, more than anybody else, have the understanding of root causes and probabilities that a particular output might be bad, and have more than sufficient resources to fix this, if it's not an intractable problem. So Occam's Razor says that (currently) this is still an intractable problem. |