Top line difference is speed. The impact of speed is deeper and gets felt over time.
Traditional research publications are no angels. They gatekeep research, and also allow financial incentives to drive them to publish junk with their stamp on it.
But a flood of papers doesn’t actually mean more knowledge. In bypassing this route entirely, AI has swiftly lost the ability to engage with itself as a field. And the cost of that is only beginning to be felt.
That pesky “scientific method” bothers these corps sometimes, so one can play fast and loose with numbers and conclusions for all sort of reasons. No need to worry about reproducibility, rigor, scrutiny, etc. not good for IPO and stocks.
No traditional research is mostly done by post-docs and have a phd level of education rather than a tech bro that passed leetcode. That seems like a good bar to have; not too mention the whole peer review thing, hard to really understand anything if you purposely withhold it and tell people to kick rocks.
Eh. Peer review isn't central to science. The point of peer review is: you have a claim someone makes; if it's been peer-reviewed, you can build further research on it without having verify the claim yourself. If you're building technology directly, any result you use you're going to verify inherently, because it either works as advertised or your tech doesn't work. The external verification doesn't add that much in that case.
Peer review isnt central, but it doesn't try to ensure a method is being followed.
If you are spending time, money, and risk based on a claim, verification still helps to cull info that is deceptive or done with methods so shoddy it might as well be lying.
Sure, you can verify through your failures, but that isnt great. You company goes belly up and you can confidently say the blog was bullshit.
And what's the point of claiming there are peer reviewed reproducable studies that show this, if you don't bother citing those studies for your peers to review?
Traditional research publications are no angels. They gatekeep research, and also allow financial incentives to drive them to publish junk with their stamp on it.
But a flood of papers doesn’t actually mean more knowledge. In bypassing this route entirely, AI has swiftly lost the ability to engage with itself as a field. And the cost of that is only beginning to be felt.