[aithrowawaycomm]

It is not a high standard, I am sure you could train a chimp to pass this test[1]. If you know how to use a standard coffee maker and live in a typical American home, and the test is done in an typical American home with a standard coffee maker, you can definitely pass this test 100% of the time.

I understand that many people don't live in America and don't know how to use a coffee maker. That is 100% irrelevant. There is a frustrating tendency in AI circles to conflate domain knowledge with intelligence, in a way that invariably elevates AI and crushes human intelligence into something tiny.

[1] The hard part would be psychological (e.g. keeping the chimp focused), not cognitive. And of course the chimp would need to bring a chimp-sized ladder... It would be an unlawful experiment, but I suspect if you trained a chimp to use a specific coffee maker in another kitchen, forced the chimp to become addicted to coffee, and then put the animal in a totally different kitchen with a different coffee maker (but similar, i.e. not a French press), it figure would figure out what to do.

[sillysaurusx]

"locate the filters, locate the coffee mugs, locate a measuring spoon" in a random house in America is a very high standard. We’ll have to agree to disagree on that. If you teleport me into a random house, I’ll likely spend at least an hour trying and failing at that task, and most of their cabinets and drawers will be open by the end of it.

It also excludes corner cases like "what if they don’t have any filters"? Should the robot go tearing through the house till they find one, or do nothing? But what if there were some in the pantry — does that fail the test? There’s all kinds of implicit assumptions here that make it quite hard.

[fragmede]

and what if there's only a Nespresso machine, a Keurig machine, instant, a french press, a moka pot, or a cappuccino machine (we can argue if an americano is actually coffee, but if that's what the house has, and no drip machine + accoutrements, you're not getting anything else)? Human or bot, that's a lot of possibilities to deal with, but for a bold human unfamiliar with those, they're just a YouTube video away (multiple ones if it's a fancy cappuccino machine). Until AI can learn to make coffee or change an oil filter on a 1997 GMC from watching a YouTube video, it'd be hard to consider it human-grade, even if it has been trained on all of YouTube, which assumedly Google has done. There are certainly things people do on YouTube that I couldn't do after a lot of intense practice, though, so I'm not totally convinced that's the right standard. It doesn't cost millions of hours and dollars of training and fine tuning time for me to, say, be able to tie a bow tie from a YouTube video though, even if it does take me a couple of tries.

[MostlyStable]

It probably shouldn't continue to surprise me how often people's "AI benchmarks" exclude a significant fraction of actual, living, humans from being "human-grade".

[EncomLab]

You can't honestly claim that it would take you an hour to accomplish such a high probability task - have you never visited the house of a friend or family and had to open a few cabinets to find a water glass or a bowl or a spoon?

As for the point of corner cases being hard - I mean that's the point here, isn't it?