[comicjk]

Stars produce oxygen (along with all elements up to iron) during nuclear fusion. If the star is big (several times bigger than our sun), it eventually explodes and the oxygen then winds up in interstellar gas and dust. The oxygen reacts with hydrogen to make water. Eventually you wind up with a lot of dirty snowballs floating around (comets).

Oxygen is pretty common and hydrogen is everywhere, so water (as ice) is not scarce in the universe. The only place where water is uncommon is near a star, like us, where the water boils off into space unless a planet has enough gravity to hold it in.