[galaxyLogic]

Good definitions. Would you then agree that when we reach the point where AI can improve itself without our help, it is still possible to make predictions, about it?

I think we are already in the stage where AI can and does improve itself. But why should this stage be called "Singularity"? Like a Black Hole? That sounds like hype to me.

When AI can improve itself, wouldn't it still be able to explain to us how it has improved itself? If it can not it still has a lot of improvement to do.

Or are we saying that some things are "unexplainable" and AI will discover such things without being able to explain to us what they are? That sounds like mysticism, or hype to me. Or religion. We can not explain God, right?

[XorNot]

The history of predicting innovation even a decade out is full of misses, even then the actors involved were all human.

It is significant that if machine-intelligence can improve itself, then there's obviously a transition point at which the bulk of innovation shifts from being done by human minds to being done by machine minds - and if nothing else that's a "phase" transition in technological progress, namely that the future of technological development is no longer the product of human innovation but of our created machines, hopefully acting according to our desires.

The technological singularity is that: the pace of innovation separates from the ability of human minds to keep up with understanding it. We can barely predict our own innovation: we would be unable to usefully predict innovation happening by alien intelligences operating faster then we can think.

Now of course where that line exists, or if it's even plausible, are separate questions (there's a reasonable argument that physical manufacturing is likely to be a serious limiter for a good long time still at the very least).