[wccrawford]

You have to be very careful when you start offering real rewards. People will game the system for free, but nothing like the level of gaming that goes on for a reward. You'll end up with absolute crap.

And don't assume that people 'working for free' are getting nothing of it. They obviously enjoy it, or they wouldn't do it. Offering a tangible reward will change their motivation.

The result of both of those is often losing your best, most loyal users.

Be very careful.

[PaulHoule]

I've done some projects on Mechanical Turk (spent enough to get an account rep!) and I can definitely say that offers of money will bring in people who will (1) rip you off or (2) do the minimum work required to get paid.

The type (1)'s are easy to deal with because there are ways to check people's work. They just don't get paid. The type (2)'s are harder -- there's a population of Turks that I call Superturks that spend a lot of time Turking. If you make a histogram of how many HITs everybody did, they'll be the people who did the most.

The trouble with Superturks is that the quality of the work they do is worse than average, usually just good enough that you wouldn't feel comfortable rejecting their work. I know I could get better quality if I banned them, but I find it hard to do.

[sneak]

Reject mediocrity in all aspects of life. Spend money on good tools, and shun anyone who makes less-than-good tools.

[tehwalrus]

indeed. this: http://www.amazon.com/Reality-Broken-Games-Better-Change/dp/... cites research that shows that people work waaay less hard on something when monetary reward is offered, than when it's just for free.