Yep; it’s obviously not a deal breaker/winner, but we definitely notice. I once joked that we ought to mark up candidates with LaTeX CVs, but knock them slightly down if they stuck with Computer Modern :)
After a while you can just spot it, LaTeX has a much higher quality typesetting and layout engine than Word or other WYSIWYG on-the-fly editors.
The easy way is usually to spot some of the no-no's of layouting; single words at the last line of a paragraph, sentences spanning over page breaks, spacing in block paragraphs, etc.
It's easiest with letters or invoices with some text, where the spacing and word placing is most obvious but you can usually tell in CVs with less text too.
Of course, sometimes Word gets lucky and does a proper layout, but I've found that it's rare that I get to see properly layouted word documents.