Your example is wrong, I think: My eyesight can't tell me about the truck around the corner and I only know it when I hear it but that doesn't mean that I can see it next time. In comparison, when you "see" the polyphony you are one step closer to "hearing" it the next time.
For instance, your eyesight is terrible at telling you a furniture truck is coming around the corner.
If you were good at separating the individual elements of music without much effort, perhaps music wouldn't be nearly as appealing. Who knows?