[skybrian]

No, it's more like getting bad advice from Stack Overflow or Reddit.

Usually you notice it's broken because it doesn't work.

There may be some advice that seems to work, but has subtle issues. (For example, race conditions.) Having more experience can help in spotting these problems.

Either way, copying code blindly isn't a good idea. In a professional situation, better testing and code review can help.

[dgroshev]

Yes, but collectively we understand that SO or Reddit are unreliable, plus it's usually one-way and we don't see people posting bad advice defending it just for us. LLMs, on the other hand, are extremely convincing, while being even less trustworthy than SO/Reddit.

Just make a mental experiment and imagine that every comment here praising LLM coding skills is actually about copypasting stuff from SO. Does it not seem alarming to you?

[skybrian]

I don't think it's any more alarming than people posting that they find Stack Overflow useful sometimes. That's why it became popular, right?

Similarly for searching the Internet. Internet searches are potentially dangerous if you can't tell good answers from bad. But if you treat it as a source of hints that you need to verify yourself, it's a pretty good source of hints.

Many of us have learned quite well that LLM's are unreliable. Someone posts about the problem whenever it comes up. It's something close to conventional wisdom.

It's still surprising the first time you see it hallucinate, though. You have to see it for yourself. Once you've seen it a few times, it isn't so convincing anymore.