[brainsareneat]

I don't understand how this is a response to the above point. Kaggle competitions and academic research are on a different timescale and have different goals. Most of the algorithms used by Kagglers were designed by academics - and those academics were rewarded for producing those algorithms with grants from both for-profit and non-profit organizations. Both individual and deeply collaborative endeavors have a place in capitalism and contribute massively to economic growth and scientific progress.

[marshallp]

I understand what you're saying, but I'm skeptical that acdemic research is all that it's cracked up to be. Studies are always popping up about how most scientific findings are fraud. And then there's things like the X prizes and darpa challenges which really show who's right and who's wrong and what's actually possible. I'd much rather see open competitions used than grants. Kaggle can be that "central" competition for the machine learning community.

[imgabe]

But an open competition doesn't actually fund anything. The money is awarded after all of the work has been done. Somebody has to pay to feed and house scientists while they're working on something, as well as for lab equipment and support staff, etc, etc. An open competition solves none of those problems.

Furthermore, a competition only compensates the winner. That's all well and good for competitions, but sometimes science advances as much by finding things that don't work (and why they don't work) as by finding things that do.

[marshallp]

There are probably solutions to all the above. "Tapered" prize giving (even 100th place gets a small prize), more frequent changeovers (rather than tenure and multiyear student commitments, have monthly or weekly reviews about wh stays and who leaves, just like in the private sector). Anyway, I acknowledge your points, I just think the present system isn't necessarily the perfect system.