Because the mathematics is basically distilled, formalized art of precise thinking. It's the art of manipulating and morphing mental models. It's as much about numbers as astronomy is about telescopes ;).
Higher math requires a great deal of lateral thinking. Also, music an math are often closely related but I don't see the connection between music an say computer vision.
Computer vision practitioners do mostly math, are we calling them computer scientists also? Sheesh :). Whenever I get in the same room with one, we are talking completely different languages (and we do have a sizeable computer vision team in our lab).
I heard that computer science was more of a subset of music than math.