[qsera]

>I meant that humans are vulnerable to malicious input too.

No they are not. Social engineering won't work on a human security expert who knows and understands the implications of the information they are giving away. Your analogy is pointless.

[jcalx]

> Social engineering won't work on a human security expert who knows and understands the implications of the information they are giving away

Social engineering, like prompt injection, is a context attack — easy to spot if you're ready for it, but harder in different circumstances (rushed, panicked, tired, having a bad day, etc.).

Troy Hunt (security consultant, creator of HaveIBeenPwned) and Cory Doctorow have both been successfully phished [0][1]. They're both tech- and security-savvy people who "should have known better" but it happened to them anyway. But maybe you're different... you'd never fall for an online scam, right? [2]

[0] https://www.troyhunt.com/a-sneaky-phish-just-grabbed-my-mail...

[1] https://doctorow.medium.com/https-pluralistic-net-2025-04-05...

[2] https://news.harvard.edu/gazette/story/2024/09/youd-never-fa...

[qsera]

> easy to spot if you're ready for it

Are you serious? LLMs, being a computer program, should always "be ready"?

Unless you want to also claim that LLMs can be rushed, panicked, tired or can have a "bad day"!

Jesus! The mental gymnastics people will go through to justify LLMs is just absurd!

[jcalx]

To be very clear — I was specifically responding to "social engineering won't work on a human security expert". It can, and has. People are not infallible, and a "that would never happen to me" mindset (1) gets people phished when they think they're too smart for it and (2) is a pet peeve of mine and so sometimes I can't resist pointing that out.

Largely agree with you otherwise, not sure why you read my comment as mental gymnastics to justify LLMs. I don't think that they have an internal emotional state that can feel rushed, panicked, so on. They do — superficially — "respond" to such cues in language, which is why they can be "threatened" [0] and "flattered" [1]. But without an internal theory of mind, LLMs do this sycophantically without any internal model of the world (hence the quotes above, to avoid fully anthropomorphizing their behavior).

The only parallel I'm drawing is that both humans and LLMs can be coerced into unintended behavior via language ("social engineering" and "prompt jailbreaking" respectively), and those attacks are more effective if an attacker is allowed to control more "context", even if the underlying mechanism of why those attacks work is completely different.

[0] https://arxiv.org/pdf/2507.21133

[1] https://arstechnica.com/science/2025/09/these-psychological-...

[maybe_pablo]

Sure they are, if the human expert follows instructions from a manager or a client, if they are of utility to anybody, then they are vulnerable to social engineering and malicious input. An attack may be easy or hard depending on the expert's training, but nobody is flawless.