[pesenti]

Our intelligence isn't task specific, but that doesn't mean it can solve any problem. It's actually full of biases and very optimized for our survival (vs being a general problem solver). It's ok to talk about more or less narrow/general tasks/intelligence. But what threshold of generality is "general"?

And the problem is that once people assert this "absolute" level of generality, they assume it can do anything, including make itself more intelligent.

[cmsj]

I don't think it's right to suggest that an absolute level of generality would be necessary for that kind of self-improvement.

If we assume a future where humans are able to create a human-level AI, then it would have at least two substantial advantages over us:

* It would probably have substantially more insight into how its "brain" works than we have of ours, because it would know how we created it. This suggests it could at least make small improvements.

* Unlike our relatively fixed brains, it would be able to remake itself over and over, either very quickly, or at least over comparatively vast timescales.

The obvious conclusion from those two factors is that it would likely be able to start at human-level, but rapidly accelerate up a curve and go far beyond our intellect in probably a lot less time than it took for evolution to come up with us.

[psyc]

I would add that it wouldn’t tire or bore, wouldn’t make trivial errors, and wouldn’t suffer from poor recall.

[psyc]

Yet humans realized we are biased and devised ways to mitigate that. It still sounds like you’re referring more to our basic goals than to our faculties. I agree that the word general is fuzzy, but to say we do not do general problem solving seems incorrect.

Aside, but a long time ago, Yudkowsky wrote that an AGI should be able to derive general relativity from a millisecond video of an apple falling. Later, he took to calling them optimization processes. Say what you will about the fellow, he has a way with words and ideas.