Yep; it’s obviously not a deal breaker/winner, but we definitely notice. I once joked that we ought to mark up candidates with LaTeX CVs, but knock them slightly down if they stuck with Computer Modern :)
After a while you can just spot it, LaTeX has a much higher quality typesetting and layout engine than Word or other WYSIWYG on-the-fly editors.
The easy way is usually to spot some of the no-no's of layouting; single words at the last line of a paragraph, sentences spanning over page breaks, spacing in block paragraphs, etc.
It's easiest with letters or invoices with some text, where the spacing and word placing is most obvious but you can usually tell in CVs with less text too.
Of course, sometimes Word gets lucky and does a proper layout, but I've found that it's rare that I get to see properly layouted word documents.
I'm sure having a LaTeX CV correlates well with other skills which you might deem desirable and, if so, it seems like a perfectly reasonable indicator to use. It shouldn't be used exclusively, but nothing should be.
I must say, I was happy to come across this after putting in the not insignificant effort of converting my resume into a nice looking LaTeX document: http://stevehanov.ca/blog/resume_comic.png
Yes, it's definitely a shibboleth. IMO it's a full duplex signal. When I notice that an interviewer noticed, it can make me more interested in an offer from that place if I think I'll be working with that/those interviewer/s.
