[psb31]

It’s a great question and a tough problem. We have written a little bit about the problem of ‘slackers’ (participants with low attentiveness and engagement) [1,2]. The things we’re doing right now to address this problem is to 1) test for attentiveness and engagement before participants take part in real studies, 2) distribute surveys more broadly to reduce the % of “professional survey takers” in each study and 3) educate researchers about ways to use good attention and engagement tests [3] in their studies. We can then feed this data back into our system so we can iteratively improve on data quality.

I can’t go into too much detail about the attention and engagement checks we have built in, but if you sign up as a participant and pay attention you might spot some. In the long run, we think good feedback systems and high trust will be key so that our pool iteratively gets better and participants don’t feel as incentivized to cheat. It’s key to make sure that participants feel that their high effort responses get fairly rewarded (both financially and with social appreciation).

[1] https://blog.prolific.co/how-to-improve-your-data-quality/

[2] https://blog.prolific.co/bots-and-data-quality-on-crowdsourc...

[3] https://researcher-help.prolific.co/hc/en-gb/categories/3600...