[wdewind]

I've used Feedback Army twice so far in the 25-50 range. I really want to love your product, and it's super well designed and easy to use, but the people on Amazon Turk just suck IMO. The feedback I got was completely generic and they seemed to be trying to answer all my questions the way I "wanted" them to - this seems to be a cultural thing in East Asia. It was very frustrating.

I understand there is some give and take with how I ask my questions and analyze the results (ie: what you put in is what you get out to a certain extent) but it would be awesome to be able to pay more and limit your users to specific countries or regions (if this means moving off Amazon Turk so be it. I would pay much more money than you are currently charging for a higher quality service).

[aantix]

I've solicited feedback for RunFatBoy.net directly from Amazon. I always ask them specifically what they don't like about the site. I also make a statement along the lines of

"It's OK to be negative; I am looking to improve my site and any criticism is welcome. You actually have a chance at a bonus if your criticism helps to improve my site."

Sadly, some people need permission or an incentive before they feel comfortable giving an honest answer.

[henryprecheur]

Yes, the answers' quality seemed very variable: the most critical answers were the most useful. I'd also be willing to pay a premium to get reviews from the "meanest" soldiers ;-)

[jasongullickson]

"Feedback Drill Sergeant" perhaps?

[huhtenberg]

My sentiments exactly.

http://news.ycombinator.com/item?id=1816884