[Chupachupski]

One approach is to get each hit done by two (or three) regular MTurkers and crosscheck the results. I've seen this done with "human OCR" type tasks.