You'd be hesitant to trust a brand if it can't keep consistent styling. Branding helps users identify a brand and believe it or not the aesthetics of a brand make a great deal of impact on consumers.
As others have said, your point comes across as "let's remove design who cares" because design and human computer interaction roles stopped where your understanding ends. Everything looks the same to you after all (it doesn't, you just haven't noticed it affecting your decision making).
Interesting how you seem so sure about what matters to other people, when the reality is that anything matters that people say matters to them. If people care about fonts, they do care about fonts.
If I follow your train of thought to its logical conclusion, nothing matters. Which is correct, but sort of pointless to state. On top of this nothingness, we typically stack personal preferences.