|
|
|
|
|
|
by BeefWellington
512 days ago
|
|
|
> The "everybody does it" argument is a classic rationalization that doesn't actually justify anything. I'd argue here the more relevant point is "these specific people have been shown to have done it before." > The whole comment reads like someone who has picked up some ML terminology but lacks fundamental understanding of how research evaluation, technical accountability, and institutional incentives actually work in the field. The dismissive tone and casual accusations of misconduct don't help their credibility either. I think what you're missing is the observation that so very little of that is actually applied in this case. "AI" here is not being treated as an actual science would be. The majority of the papers pumped out of these places are not real concrete research, not submitted to journals, and not peer reviewed works. |
|
|
This is itself a slippery move. A vague gesture at past misconduct without actually specifying any incidents. If there's a clear pattern of documented benchmark manipulation, name it. Which benchmarks? When? What was the evidence? Without specifics, this is just trading one form of handwaving ("everyone does it") for another ("they did it before").
> "AI" here is not being treated as an actual science would be.
There's some truth here but also some sleight of hand. Yes, AI development often moves outside traditional academic channels. But, you imply this automatically means less rigor, which doesn't follow. Many industry labs have internal review processes, replication requirements, and validation procedures that can be as or more stringent than academic peer review. The fact that something isn't in Nature doesn't automatically make it less rigorous.
> The majority of the papers pumped out of these places are not real concrete research, not submitted to journals, and not peer reviewed works.
This combines three questionable implications:
- That non-journal publications are automatically "not real concrete research" (tell that to physics/math arXiv)
- That peer review is binary - either traditional journal review or nothing (ignoring internal review processes, community peer review, public replications)
- That volume ("pumped out") correlates with quality
You're making a valid critique of AI's departure from traditional academic structures, but then making an unjustified leap to assuming this means no rigor at all. It's like saying because a restaurant isn't Michelin-starred, it must have no food safety standards.
This also ignores the massive reputational and financial stakes that create strong incentives for internal rigor. Major labs have to maintain credibility with:
- Their own employees.
- Other researchers who will try to replicate results.
- Partners integrating their technology.
- Investors doing technical due diligence.
- Regulators scrutinizing their claims.
The idea that they would casually risk all that just to bump up one benchmark number (but not too much! just from 10% to 35%) doesn't align with the actual incentive structure these organizations face.
Both the original comment and this fall into the same trap - mistaking cynicism for sophistication while actually displaying a somewhat superficial understanding of how modern AI research and development actually operates.