[saberience]

lol 100% approval rates? No they don’t.

Also mammals? What mammals could even understand we were giving it a test?

Have you seen them or shown them to average people? I’m sure the people who write them understand them but if you show these problems to average people in the street they are completely clueless.

This is a classic case of some phd ai guys making a benchmark and not really considering what average people are capable of.

Look, these insanely capable ai systems can’t do these problems but the boys in the lab can do them, what a good benchmark.

[nipah]

quoting my own previous response: > Also, I mentioned mammals because those problems are of an order that mammals and even other animals would need to solve in reality for a diversity of cases. I'm not saying that they would literally be able to take the test and solve it, nor to understand this is a test, but that they would need to solve problems of similar nature in reality. Naturally this point has it's own limits, but it's not easily discarded as you tried to do.

---

> Have you seen them or shown them to average people? I’m sure the people who write them understand them but if you show these problems to average people in the street they are completely clueless.

I can show them to people on my family, I'll do it today and come back with the answer, it's the best way of testing that out.