Hacker News new | ask | show | jobs
by sillysaurusx 469 days ago
"locate the filters, locate the coffee mugs, locate a measuring spoon" in a random house in America is a very high standard. We’ll have to agree to disagree on that. If you teleport me into a random house, I’ll likely spend at least an hour trying and failing at that task, and most of their cabinets and drawers will be open by the end of it.

It also excludes corner cases like "what if they don’t have any filters"? Should the robot go tearing through the house till they find one, or do nothing? But what if there were some in the pantry — does that fail the test? There’s all kinds of implicit assumptions here that make it quite hard.
2 comments
fragmede 469 days ago
and what if there's only a Nespresso machine, a Keurig machine, instant, a french press, a moka pot, or a cappuccino machine (we can argue if an americano is actually coffee, but if that's what the house has, and no drip machine + accoutrements, you're not getting anything else)? Human or bot, that's a lot of possibilities to deal with, but for a bold human unfamiliar with those, they're just a YouTube video away (multiple ones if it's a fancy cappuccino machine). Until AI can learn to make coffee or change an oil filter on a 1997 GMC from watching a YouTube video, it'd be hard to consider it human-grade, even if it has been trained on all of YouTube, which assumedly Google has done. There are certainly things people do on YouTube that I couldn't do after a lot of intense practice, though, so I'm not totally convinced that's the right standard. It doesn't cost millions of hours and dollars of training and fine tuning time for me to, say, be able to tie a bow tie from a YouTube video though, even if it does take me a couple of tries.
link
MostlyStable 469 days ago
It probably shouldn't continue to surprise me how often people's "AI benchmarks" exclude a significant fraction of actual, living, humans from being "human-grade".
link
EncomLab 469 days ago
You can't honestly claim that it would take you an hour to accomplish such a high probability task - have you never visited the house of a friend or family and had to open a few cabinets to find a water glass or a bowl or a spoon?

As for the point of corner cases being hard - I mean that's the point here, isn't it?

link