Sounds a lot like what I've done...I'm 50/50 at best. Vote streaks are just session-based unless you are logged in, but since I have enabled voting I'm at like 54.13% with about 300 votes or so. Not too impressive, even after looking at mugshots all week...
Hah yeah I definitely haven't warmed up yet. I think my longest streak has been 7 or 8...there's always a match-up of two scary people that just throws me for a loop.
For a while I was convinced that a distinct mustache (vs general unshavenness, etc) was a very strong indicator. It's amusing to watch yourself try to develop heuristics.