I would suggest being a bit patient with them.
I wish they were better prepared for this surge in popularity, but now that we are here, it will probably take weeks for them to iron out all the core issues to achieve proper scalability.
If they are still having issues a month from now, then I would be concerned.
Unless you're self hosting or using an alternate server Matrix.org will run into much of the same. Delivery can already be quire delayed occasionally on the main server, and it's barely popular in terms of big picture right now.
If they are still having issues a month from now, then I would be concerned.