Hacker News new | ask | show | jobs
by lilsunnybee 4359 days ago
It's been over a year and a half since i've spent any serious time on Mechanical Turk trying to supplement low income, so some of my experience might be dated. With a bit of a learning curve it's plenty doable to find decently paying jobs in transcription, writing, etc. on MT, especially if you have a strong grasp of the English language. But even with higher-skilled, higher-paying jobs on that platform, the overall experience of working on MT can often end up being plenty demoralizing and unfair in many many ways, such as qualifying for skilled work and also being compensated for work performed.

The most frustrating jobs i've ever performed on the platform were usually approx 5-10 minute tasks that seemed to be decently paying and straightforward, asking you to categorize items based on some arbitrary but seemingly simple system, but later all 5 or more submissions you might have made could be blanket denied / payment refused. In that case not only do you not get payed, but having HIT submissions rejected really seriously hurts your reputation and what jobs you're allowed to apply for on MT; often many more high paying jobs on the system use high acceptance rate and flawless work history as the lowest bar for entry, and so MT workers can be screwed over incredibly by having work rejected through no fault of their own.

To not get seriously screwed newcomers are somehow supposed to know which types of HIT's to definitely avoid that could kill their rating, but without enough information to actually make that judgement. A task might seem easy and give the illusion that it's okay to send 5 or 10 submissions in a row, only to later have them all rejected and serious reputation damage done.

I think it's interesting but also really sad how this mirrors a lot of the unfairness and power imbalance in the larger working world as well. Reputation is everything, but if you just happen to get unlucky to start out your career working for clueless bosses / clients with unrealistic expectations, there's a good chance future job and career prospects are going to be seriously hindered, if not completely derailed.

Transcription work was usually very well paying i found, but doing one long transcription task for a $30 payout is a huge risk the way things were set up. Never mind that you worked for 4 or 5 hours on it; whoever assigned the job can reject it for whatever random reason they want and you get nothing. As nice as it can be to earn a nice wage for more high-skilled tasks, the possibility of being denied any payment for a days work with little recourse is really frustrating and demoralizing; the way Mechanical Turk is structured if anyone's taking a negative financial hit for work done, it's almost without fail going to be the worker.

Though this CMU professor obviously put some work into this write-up, i don't see how somebody can possibly get an accurate picture of what it's like trying to earn an income on one of these platforms by just devoting 4 hours and extrapolating based on that really limited experience. It is really easy to miss all but the most obvious and glaring problems that way, just from not sticking around long enough to even run across them.

There are plenty of MT workers who have spent much more substantial amounts of time on the platform, that are much better able to communicate benefits and pitfalls for the everyday worker. They might not be professors at prestigious universities like CMU, but having a decent amount of hands-on experience should really be the low bar for discussing pros/cons of the platform seriously; just as one example assumptions about longer-term take-home pay in the write-up were pretty naive: many better paying tasks (especially web browsing tasks and surveys paying more than a pittance) are often in limited supply and cannot be repeated either at all, or only a few times per day / week, etc. Even the best employers providing the most decent fairly-compensated work come and go, making a steady healthy income sometimes very difficult to achieve even in the best of circumstances.

Mechanical Turk has experienced a lot of success from the start largely because they created a service that makes it so easy for virtually anybody with internet access and a basic computer, to work from home and perform simple tasks / other types of work when and wherever they want, and actually get paid for it.

By any decent standards of developed-world countries the pay is shit though, and worker protections and benefits are nonexistent; workers can be denied payment and even have their reputation ruined for no good reason, with little in the way of an appeals process. But even now services like MT are still a pretty new thing. Maybe in 10 or even 20 more years most of the bugs will get worked out, and workers of every skill level can be a on more equal footing when negotiating with employers, settling disputes etc.

3 comments

If there's no real appeal process, what reason is there for any MT employer to make any payouts at all? Honesty and not realising this about the system?
Maybe i read it wrong, but from the article it sounds like the Amazon MT team is at least trying to do more in the way of preventing unnecessary rejections and bad faith task-giving. Not sure if any sort of appeals process was ever put in place, but i really doubt it, since that would create a new non-trivial administrative burden on Amazon that is likely much larger than they are willing to handle.

Seemingly arbitrary rejections always seemed to be a much bigger problem by my estimation, especially for workers already somewhat invested in the system, and needing to maintain approval ratings to even qualify for higher-paying types of work.

If Amazon hasn't implemented anything like a way of expunging old rejection records, it would be a huge benefit to their workers to have some sort of reasonable, semi-automated way of doing so, at least after enough time has passed, or enough more recent positive approvals have occurred.

I'm really out of the loop though, and don't know much about the current state of things. It would just be really nice to have some more in-depth write-ups on the MT worker experience. The article was great but from first-hand experience i know there is a lot more ground to cover.

Senior project / undergrad thesis anyone? Researching / investigating MT from a user interface, user interaction design side of things, giving recommendations for improvements to the worker experience, for HIT providers, etc. would be very interesting and potentially very beneficial for the platform.

There's an unofficial ratings system for those who post the jobs[1] that most people who use MT with any sort of regularity (including myself back in the day) use. If a requester consistently fails to pay out, they'll stop getting as many workers who are willing to work on their tasks.

[1]turkopticon.ucsd.edu

That's a great point! Though still really hoping the default experience has improved in the past year or so as well. Not every new or even semi-experienced user is going to be savvy and aware of external rating systems, or other peripheral tools to help improve the worker experience. I know i wasn't (first time hearing about turkopticon, sadly -_-), but that doesn't reflect at all on how qualified someone is to do the mostly very simple tasks available to work on.
I'd be interested to see how you all did if using either end of Pay4Bugs ( https://www.pay4bugs.com )

I bet you'd learn some great stuff and see some crazy software

Neat! I haven't heard of it before but will definitely check it out. Glad to see more and more options popping up for this sort of flexible, easy payment type work. This kinda thing can do a lot to help people already below, or hovering right above the poverty line, since unforeseen expenses, and not being able to get new sources of income started in time is one of the biggest most daunting problems to have to deal with.
This is an excellent comment.

Looking at your posting history, I can see that you often take the time to write thoughtful comments. Please keep it up.

Thanks. :-)