[lambdatronics]

Journal referees are about as far from "real world feedback" as you can get.

There's an interesting clash of incentives -- the grant-funding agencies optimize for stuff that is low-risk, but they claim to want highly-novel work, which also what gets published in better journals. So often what you must do is dress up incremental progress as revolutionary progress. It's soul-sucking.

[skosuri]

What would you propose as an alternative to peer-feedback? I really enjoyed academia and didn't find it soul sucking. Does it suck to get rejected sometimes? Yes. Does the work eventually get out; yes? With pre-prints these days, most of the people that are in the field already read the paper before acceptance if it's good. So the journal/conference name itself is largely a symbolic judgement of your peers.

[lambdatronics]

I don't think we should go without some form of review -- just that other academics are not representative of the real world. Perhaps a system where the 'relevance' claims are evaluated by industry reviewers, and the scientific claims are evaluated by academics?

Post-publication (I'm counting arxiv here as publication) review is an improvement. My branch of science doesn't use arxiv to any real extent, unfortunately.

Edit: I left a word out.