You can't really have one without the other because mathematics describes lots of universes, not just ours. If you embrace mathematical universe you discard the notion of any particular concrete "reality" - our universe is not the single privileged "real one" just because we happen to be in it. In fact that's pretty much all there is to the idea.
Now throw in the fact that actual physics' best current guess is our universe is in actual fact constantly 'splitting' (although the term is misleading because the process is continuous, not discrete - the universe is constantly diverging from itself, everywhere) - there "really" is a multiverse. This lends a sort of philosophical weight to mathematical universe by rejecting a single, privileged version of events - but it also presents you with some weird dilemmas with subjectivity. To wit - if a quantum, universe-splitting choice is made, and given that both versions of events will be witnessed by some version of you - how does that give rise to the phenomenon of subjective probability i.e. your personal anticipation of witnessing a particular version of events? It seems there is at least some sense in which one universe is more "real" than another - you're more likely to end up there!
Now throw in the fact that actual physics' best current guess is our universe is in actual fact constantly 'splitting' (although the term is misleading because the process is continuous, not discrete - the universe is constantly diverging from itself, everywhere) - there "really" is a multiverse. This lends a sort of philosophical weight to mathematical universe by rejecting a single, privileged version of events - but it also presents you with some weird dilemmas with subjectivity. To wit - if a quantum, universe-splitting choice is made, and given that both versions of events will be witnessed by some version of you - how does that give rise to the phenomenon of subjective probability i.e. your personal anticipation of witnessing a particular version of events? It seems there is at least some sense in which one universe is more "real" than another - you're more likely to end up there!