[lagrange77]

I'm not just imagining self-improvement in the sense, that it optimizes its design to become more efficient or powerful by a few percent. A system, that can 'think out of the box' may come up with some disruptive new ideas and designs.

Just a thought: Isn't lacking intellectual capacity, what keeps us from understanding, how the human brain actually works? Maybe an AGI will eventually understand it and will construct the next AGI generation with biological matter.

[treesprite82]

I think diminishing returns applies regardless of whether it's improving itself through optimization or breakthrough new ideas. There's only so much firepower you can squeeze out if everything else were to remain stagnant.

Like if all modern breakthroughs and disruptive new ideas in machine learning were sent back to the 70s, I don't think it'd make a huge difference when they'd still be severely hamstrung by hardware capabilities.