[Timwi]

I also believe that AGI is not a thing, but for different reasons. I notice that almost everybody seems to implicitly assume, without justification, that humans are a GI (general intelligence). I think it's easy to see that if we are not a GI, then we can't see what we're missing, so it will feel like we might be GI when we're really not. People also don't seem interested in justifying why humans would be GI but other animals with 99% of the same DNA aren't.

My main reason for thinking general intelligence is not a thing is similar to how Turing completeness is not a thing. You can conceptualize a Turing machine, but you can't actually build one for real. I think actual general intelligence would require an infinite brain.

[Mordisquitos]

> I notice that almost everybody seems to implicitly assume, without justification, that humans are a GI (general intelligence). I think it's easy to see that if we are not a GI, then we can't see what we're missing, so it will feel like we might be GI when we're really not.

That's actually a great point which I'd never heard before. I agree that it's very likely that us humans do not really have GI, but rather only the intelligence that evolved stochastically to better favour our existence and reproduction, with all its positive and negative spandrels[0]. We can call that human intelligence (HI).

However, even if our "general" intelligence is a mirage, surely what most people imagine when they talk about 'AGI' is actually AHI, as in an artificial intelligence that has the same characteristics as human intelligence that in their own hubris they believe is general. Or are you making a harder argument, that human intelligence may not actually have the ability to create AHI?

[0] https://en.wikipedia.org/wiki/Spandrel_(biology)

[Timwi]

Yes, I do think that people usually mean AHI even they say AGI, although they don't realize it because when asked to define AGI they talk about generality and not about mimicking humans. (Meanwhile, when they talk about sentience and consciousness, they will usually only afford that to an artificial entity if it is exactly like a human, and often not even then.)

> Or are you making a harder argument, that human intelligence may not actually have the ability to create AHI?

I wasn't, but I've pondered it since you brought it up. No, I don't think it's impossible to create a greater intelligence than oneself — in fact, evolution has already done it by creating animals, including but not limited to humans. I used to think it was impossible when I pondered science fictional characters like Data from TNG, but modern LLMs show that we can create it without having to understand how it works. Data is depicted as having been engineered, but machine learning is closer to evolution than it is to engineering.