|
|
|
|
|
|
by jerf
871 days ago
|
|
|
I don't have great English words for this, but the biggest concern for me with LLMs is that of all the text generation algorithms I've ever seen, they are just fantastic at producing output whose plausibility to the human mind greatly exceeds its actual quality, the difficulty of concretely measuring either of those values notwithstanding. Note I'm not even strictly speaking criticizing the quality of the output per se. It is also a big jump over any previous technology and very impressive in its own way. It is, nevertheless, quite dangerous because the jump in the human-perceived plausibility is much larger than the quality improvement. Whereas earlier techs were obviously wrong to a human reader, in the case of code generation so obviously wrong that we never even considered using them, LLMs are extremely good at hiding the errors in the parts of the code that we are cognitively most inclined to overlook. This also has the effect of making it bizarrely difficult code to fix. How it does this I do not know. A fascinating research question for some ambitious cognitive scientist. But the signal is very strong and I don't need to wait for a paper to come out to see it. I do not think this is fundamental to AI. As I like to remind people, LLMs are not the whole of AI. They're just one technique, and one that partially for the very reason I discuss in this post, one I expect to eventually become a part of a larger system that can fix this problem at some higher level. I expect people to someday look back and laugh at us for thinking that LLMs could be used for all the things we think they can be used for. But the reasons they will be laughing are the very experience we're gathering now, and there's no skipping that phase. |
|
|