So long as they're humanely harvested? Some argue that cows are mistreated when kept producing milk as much as they are, but I haven't looked into it because I'm selfish too.
Cows are mammals. They produce milk for their young for a period after giving birth then stop, just like a human woman. Which means that for us to take their milk we have to keep them constantly pregnant.
Ask a woman (or think about it if you are one) how they would like being forcefully impregnated then having their tits constantly milked, year after year. As a bonus, the born kids are separated into girls to be milked in the same cycle and boys to be killed and eaten.
Agree if the animals are treated well; but I have a very high bar for that. I would also accept eating animals which died of old age if it could be done safely. It's easier to just round this to "vegan."