No no, I mean the theoretical analysis of neural networks. It's true that some authors attempt to do that, in fact even famous researches attempt to do this. However their arguments and analysis often breakdown in the general case and what their papers boil down to is that neural networks are good at modeling functions that they are good at modeling.
In fact the state of the field of machine learning is essentially the state of mathematics before Cauchy and Weierstrass.
What kind of analysis do you miss? There is a huge amount of research analysing all kind of aspects. See Schmidhubers overview paper oder Goodfellows Deep Learning book, or many other references to get an overview.
Of course, there are still many open questions. But I don't think that there is any aspect where there is no analysis available yet.
Also these lecture notes https://github.com/joanbruna/stat212b