[marshallp]

It's not as simple as that. It's not one poor guy looking a video feed or judging ads. It would be sophisticated system that pools together many workers in ways to get quality.

So, for example, if the factory robot knocks something over, and there are 10 different people watching a few frames of video, it's pretty likely they'd spot the mistake.

Same goes for ad critiques. You wouldn't ask "is this a good ad". You'd split it up. "Is this actor better wearing a blue shirt than a red shirt" - ask 10 people.

[spindritf]

> Same goes for ad critiques. You wouldn't ask "is this a good ad". You'd split it up. "Is this actor better wearing a blue shirt than a red shirt" - ask 10 people.

But you're still asking the "wrong" people. And you're asking, instead of observing the actual behaviour.

[marshallp]

You can have statistics figure that out for you. Are turkers good at judging ads based on past data. If not, maybe it's because you weren't presenting the right metrics for the turkers to judge and so you keep reiterating until you do. This is actually a good business for a startup, you get runaway success such as google has (the data keeps improving your lead over rivals).