[ben_w]

"No quibble" is more a mark of ASI than AGI; while both are easily coopted as marketing terms[0], "smarter than all humans" is harder to dispute than "general intelligence" (can do some graduate level work, can't drive well enough to render steering wheels obsolete, is it general?)

[0] e.g. Zuckerberg: https://www.meta.com/superintelligence/

[throwaway27448]

> "smarter than all humans" is harder to dispute

I don't think this is the case. It's not clear intelligence is a coherent concept to begin with, let alone a one-dimensional thing you can maximize. What, is it going to write stageplays that puts shakespeare to shame? Find a way to enforce world peace or end greed? I'm not at all convinced we anything more than the simple competency we expect from humans is possible. What would we look for to know it's there?

[ben_w]

> What, is it going to write stageplays that puts shakespeare to shame? Find a way to enforce world peace or end greed?

If it did, who would say "nope, still not super-intelligent"?

(I'm sure it's more than "nobody": the least realistic thing to me about my brother's LARP group was that nobody in that universe denied the gods it contained even though they'd immediately smite anyone who did that).

[throwaway27448]

> If it did, who would say "nope, still not super-intelligent"?

I'd be very happy to get better plays. I don't think we'll see this either, as this implies spending a lot on something with no return.

> (I'm sure it's more than "nobody": the least realistic thing to me about my brother's LARP group was that nobody in that universe denied the gods it contained even though they'd immediately smite anyone who did that).

That's sort of my point: judgement across history is inherently a messy process that entails consensus, which is anathema to a capital-driven world. Even the idea that there is a correct way to run society is very western and wouldn't pass earnest scrutiny. I strongly suspect that any sort of "super-intelligence" won't be useful to capital as it won't actually solve capital's problems—instead, it will reveal the contradictions inherent in a capitalist/western society and point strongly towards capital as the cancer that is crippling our planet. Hence, I don't think we will ever see it.