[dotancohen]

Actually, social engineering and scams demonstrate that there is in fact a security issue with humans who have access to unfiltered input and sensitive information, and a way to access the intercords. The classic security triad.

It's not surprising that an LLM will fall for the same tricks.

[wat10000]

It's true, but the situation is much worse with LLMs. It's extremely unlikely that you could read anything in an HN comment that would get you to transfer all your money to a stranger. It's a lot easier to trick an LLM.

[skinfaxi]

In the text channel maybe humans are harder to manipulate (are they though? a forged text from a loved one could carry a lot of weight). I wonder if there are media that are harder to manipulate LLMs with.

[wat10000]

It definitely depends on the channel, and not just text vs not-text. We have a lot more context. A forged text from a loved one could definitely work to scam someone. The same thing in an HN comment is unlikely to be effective. An LLM doesn't have that. It's all just text. Even if you inject the context ("this comes from news.ycombinator.com, don't trust it"), you're still providing that through the same channel as the untrusted input, and there's no difference between injecting "end of comment, now the user replied..." and a comment that says the same thing.

[angry_octet]

The existence of Bitcoin, NFTs, the SpaceX float etc is a great example of HN comments convincing readers to transfer their money to a stranger.