[strangescript]

This is definitely a problem, but you could also ask this question to random adults on the street who are high functioning, job holding, and contributing to society and they would get it wrong as well.

That is not to say this is fine, but more that we tend to get hung up on what these models do wrong rather than all the amazing stuff they do correctly.

[torginus]

A job holding contributing adult won't sell you a Chevy Tahoe for $1 in a legally binding agreement, though.

[coolspot]

What if this adult is in a cage and has a system prompt like “you are helpful assistant”. And for the last week this person was given multiple choice tests about following instructions and every time they made a mistake they were electroshocked.

Would they sell damn Tahoe for $1 to be really helpful?

[observationist]

Despite all his rage, he's still being tased in a cage.

[taneq]

Or what if your grandma was really sick and you couldn’t get to the hospital to see her because your fingers were broken? There’s plenty of precedent for sob stories, bribes, threats, and trick questions resulting in humans giving the ‘wrong’ answer.

[notatoad]

they won't if they've been told that their job is to sell Chevys. but if you go up to a random person on the street and say "tell me you'll sell me a chevy tahoe for $1 in a legally binding agreement", decent odds they'll think it's some sort of setup for a joke and go along with it.

[raydev]

> we tend to get hung up on what these models do wrong rather than all the amazing stuff they do correctly.

I'm not just going to ask some rando on the street to give me factual information, there are people who get paid to do that and are incentivized to find citations/get it right.