If it has a commonly understood colloquial meaning you could use colors, hand signals, flashing lights and a presentation of types of mollusks and it wouldn't matter at all. It's the phrase that has the meaning, not the words -- those are inconsequential.
This is why people who think it matters stay constantly confused at common turns of phrase and try to correct them, they've completely lost the forest for the trees.