[oncallthrow]

I suppose transistors is a bad example.

Perhaps a better analogy would be the Linux kernel. It's built by biological humans, and fallible ones at that. And yet, I don't feel the need to learn the intricacies of kernel internals, because it's reliable enough that it's essentially never the kernel's fault when my code doesn't work.

[yellowflash]

Kernel is a bad analogy, if you understand how it behaves you can understand how its built. LLMs don't have that, their behaviour is not completely defined by how they are built.

Every abstraction is leaky, its not like I have 1 in every 100 tickets I work on needs understanding of the existence of filesystem buffers, it's in the back of my mind, it's always there. I didn't read linux kernel source, but I know it's existence. LLM output doesn't have that.