| Crowdsourcing marketplaces offer unique advantages to researchers that are simply impossible to achieve with traditional methodologies, here's a few: 1) External Validity:
Today, almost all research labs in Universities have to source their participants from their respective undergraduate population. Studies suggest that the results gathered from 'convenience samples' cannot be generalized to the general population. That is, the results from experiments conducted on 19-23 year old Stanford undergraduates may not be representative of the general population (http://www.mccombs.utexas.edu/research/~/media//Files/MSB/Re...). MTurk offers researchers a more diverse subject pool in age, geography, language, and culture. (https://ccrma.stanford.edu/~jieun5/publications/2012-icmpc-m...) 2) Cross-cultural and International Studies:
Conducting cross-cultural research though traditional methodologies requires dealing with differing national requirements for protecting human subjects;variation in data collection processes in primary care practices; data transmission among participants; duties and tariffs on necessary instruments; fluctuation in currency exchange rates; incapacitation of coinvestigators; complex administration of funds; financing the additional, legitimate costs of collaboration; sustaining strong personal relationships among coinvestigators; and accepting longer time frames than would otherwise be expected. (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1466730/). The diversity on Mechanical Turk facilitates cross-cultural and international research at a very low cost (labs.yahoo.com/files/mturkmethods.pdf) 3) Faster Theory/Experiment Loop:
One implicit goal in research is to maximize the efficiency with which one can go from generating hypotheses to testing them, analyzing the results, and updating the theory. Ideally, the limiting factor in this process is the time it takes to do careful science, but all too often, research is delayed because of the time it takes to recruit subjects and recover from errors in the methodology. With access to a large pool of subjects online, recruitment is vastly simplified (labs.yahoo.com/files/mturkmethods.pdf) Traditional researchers have raised questions about the quality associated with the data collected from participants. The empirical evidence strongly suggests that the quality of data collected from mturk participants meets or exceeds the standards associated with published research (http://www.pomona.edu/Academics/departments/psychology/files...) (http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjourna...). Evidence that Mechanical Turk is a valid means of collecting data is consistent and continues to accumulate (Mason & Suri, 2012) Disclaimer: I'm a crowdsourcing technology advisor to the department of Psychology at Vanderbilt University, VizCogLab, and UVic Center for Autism Research. I'm also the co-founder and CEO of Cognilab Technologies, an online platform that allows researchers to publish their experiments on mturk. |