That's actually something we are working on. We are working with verified researchers in the field (industry and academic) to help surface good papers and foster an open discussion.
There are websites that try to do that (e.g. scirate) but the problem is that people doesn't participate much. The problem is not the website, but convincing academics to comment and vote (academics are typically starved of time, and reading a paper and writing a good coment is not easy...)
It would be nice if someone solved the problem and managed to create a working one, though.
What about collecting tweets from verified researchers? It could help getting to critical mass. You could even consider papers tweeted by researchers that are followed by your known researchers, and so on, with the right weighting (something resembling pagerank).
With the right weighting this could really boost the size and quality of your dataset.
Right now we are working on helping the community surface information and working with verified researchers to build their "curated" lists for different topics.
It would be nice if someone solved the problem and managed to create a working one, though.