[saghm]

If you change it from asking a question to giving an instruction, how many humans do you know that have trouble saying no to things that aren't reasonable? I'd argue that pretty much every human will refuse to do most things you might instruct them to do, whereas an LLM will happily attempt most things you ask them to do for you, regardless of whether they're capable of succeeding, and it's up to you to figure out if they actually did it right or not. There are tasks where this is extremely useful, but they're ones that are extremely low risk and can easily be audited upon completion. This isn't anywhere near the level of what a human is capable of.