All those points apply to XMPP too, and it's way easier to set up Prosody on a server. Are there any specific reasons why you think Matrix is the next IRC and not XMPP?
Federating is one goal but is useless without mainstream adoption.
Success is adoption. Enough users to break the network effect.
Signal is currently a good example of roughly the amount of users you need to start breaking the effect. So that's what success looks like. A very low bar version of it.
The question isn’t “If this can work with Matrix why not XMPP?”, the question is “Will Matrix have the same issues as XMPP?”