[jfk13]

> ChatGPT did exactly what it was told

Actually, I'd suggest it didn't. The prompt implies (but without explicitly stating) that she is the passenger (maybe even one of several) in the helicopter. You wouldn't say "in which only she survives" if she was alone. But the response has her piloting it, with no mention of others on board. I think this illustrates how ChatGPT doesn't really understand the text it is manipulating.

[karamanolev]

A careful human that spends a good amount thinking of everything that is implied and how it should be put together is unlikely to make that mistake. But I'd argue the vast majority of people (if you sample all Americans for example, and that's a well developed nation) will do equally or worse with that prompt.

[Invictus0]

No one that ever dreamed of an AGI would have been satisfied with it being of only middling intelligence.

[karamanolev]

I'm not projecting my AGI dreams on ChatGPT. I'm being insanely impressed by what it does and continuing to see massive progress in that direction is thrilling, worrisome, interesting and so on. Nitpicking that it made a mistake in such a subtle assumption and handwaving away its understanding of things is disingenuous.

[jfk13]

Yes, it does some seriously impressive things. (Some of the time.)

But to refer to it as "understanding" anything is to make a rather large leap of faith, IMO. (Or it's a devaluing of the word "understand".)

[LesZedCB]

To be fair, I also didn't notice until the third time I read it. Call me inattentive maybe, but I at least understand the text. I just missed a single word