[blintz]

How do people feel about the research ethics of this? Many academic studies involve the use of MTurk participants, but it seems as though they are not being paid close to a living or even fair wage ($2 an hour is abysmal!).

Should universities require that studies using MTurk pay some minimum pro-rated hourly wage? That is, if a Stanford HCI study (which seem to use MTurk often) wants workers to fill out a 15-minute survey, they would need to pay at least $8 an hour (so $2).

I’ve been to thesis defenses at Stanford where researchers have explicitly stated that they chose to use MTurk because they can pay workers far less than locally recruited participants.

[stubish]

I wonder if they actually get meaningful results from MTurk participants; you are getting a very skewed and unnatural demographic, and I don't see how any results based on feedback from MTurk participants can be applied to any normal population. Much more so than the traditional study group of students-who-need-to-pay-rent.

I don't see an ethical problem in how much you pay participants (after all, how much it pays or if it pays at all will be one of the variables the study needs to account for). The ethical problem I see is not caring if the results are meaningful, just publishable.

[mritchie712]

I'd think it's so skewed that it makes the results worthless. I wonder if anyone's done a study on it (e.g. get an actual random US sample vs. a MTurk sample and compare the results).

[phalangion]

> Much more so than the traditional study group of students-who-need-to-pay-rent.

Why do you say that? I imagine the MTurk demographic is much more diverse than "college freshman at X university"

[xienze]

> I wonder if they actually get meaningful results from MTurk participants

I bet they do. One thing not really touched on in the article is that one of the task qualifiers is “acceptance rate”, i.e., how many of your task submissions were accepted by the person that created the tasks. And that number is usually 99.9% acceptance (of _all_ results you’ve ever submitted) or higher. There is a very strong incentive to not get any kind of rejected submission, because it can quickly tank your MTurk profile and shut off lots of opportunities.

[ashelmire]

I know that we use MTurk at the Linguistic Data Consortium (and that others in NLP use it as well). I know that those involved with it put in a lot of work to make sure we get good, useful data - there are many papers and strategies on the subject of crowdsourcing.

[PopePompus]

The treatment of adjunct professors in the US tells you all you need to know about University ethics.

[phalangion]

I'm currently paying MTurkers $2 for a study that takes 10-15 minutes. That works out to $8-$12 per hour. I would never be able to recruit people to participate locally for just $2, but I can on MTurk because mine is one of many tasks they can complete in an hour.

[goldemerald]

There is an ethic commitee (the IRB) at every university that reviews all academic research involving people. In order to conduct a user study, researchers are required to compensate participants fairly. The reason you can pay MTurkers "far less" is that a study may only take 5 minutes, and researchers typically pay $10/hour. People won't participate in a study unless the minimum payment is decent. I can't imagine many people would want to take the time out of their day to show up at a lab, work on a computer task for 10 minutes, and get paid $2.

[wsc981]

2 USD per hour can be decent depending the location. Where I live in Thailand people on average make 7 USD per day.

[dsr_]

Do the people in Thailand who make $7/day generally have reliable access to the Internet for 8 hours a day, with a cost for network access and smartphone under $8/day? And a ready source of loans to cover the initial purchase? And can they read English?

[wsc981]

Yeah, internet in our remote village is quite excellent. I've worked from home with on mobile 4G internet, was good enough for video calling. And now have proper broadband in home of course.

English skills aren't great for most people, though if people really wanted to could learn for free using apps like Duolingo (there's English lessons for Thai people).

A lot of people have decent smartphones. Good 4G internet for a month cost me around 600 Baht per month (~15 USD). Sure could be expensive if people make 300 Baht a day, but if they could work through Mechanical Turk, they could quickly earn back the cost, since they would likely earn more compared to working on a farm.

Thai people like to lend money a lot. They're the most indebted of SEA when looking at loans compared to their incomes [0]. The use these loans to buy new cars, houses and phones as well.

---

[0]: https://www.chiangmaicitylife.com/citylife-articles/drowning...

[nl]

Yes.

You will probably get better English skills in Philippines or India, but the economics are similar.

[turtlecloud]

Globalization means a race to the bottom for the value of labor. $2/hr is great for a dude in India.