[nmcfarl]

So CastingWords has obviously made MTurk work and work well, but finding ways to maintain quality levels and consistency has not been an easy process. We've had to give up completely on "quick and dirty" (IE super-cheap) transcription. Instead we focus on a much higher quality product. And maintain quality by using with many many (mainly QA) steps - the shortest, simplest transcript is seen by 14 turkers - and runs through a 7 step process. And with added audio length the pipeline gets even longer.