[ivape]

That whole thing feels like a crypto coin, as in, its currency that’s worth something to just that particular group. The industry obviously doesn’t care about all these papers, so the question is, what is the social structure where these papers provide status and respect (who values their currency?).

[bonoboTP]

Science is prestigious and quick and quantifiable way to measure it are used as heuristic proxies. There are many angles to answer your question. Are you interested on the industry connection, how it translates to money, or the political aspects etc? People generally have little time for evaluation, there is an oversupply of applicants, being able to point to metrics can cover your ass against accusations of bias. It offloads the quality assurance to the peer review system. This person's work has been assessed by expert peers in 5 instances and passed to acceptance in a 20% acceptance rate venue where the top experts regularly publish. It's a real signal. They can persist through projects, communicate and defend it against the reviewers, has presented it to crowds, etc.

Its a prestige economy. There are other things too like having worked with someone famous or having interned in a top company.

[ivape]

Prestige economy is what I suspected. I recently read an AI paper that I mostly came up on a random walk, but there was a Stanford student that had already created the research paper (not exactly but more or less). In terms of “true” signal, I’d imagine that student getting reviewed as credible signals that we’re in bad shape because I can promise you I came up with the exact thesis and implementation and it was truly just common sense stuff - not research worthy.

Makes me wonder, have I turned brilliant or is it quite unimpressive out there?

I’m inclined to even suggest to you that the prestige economy started with truly prestigious research work, of which then the institutions “ordered” as many more of those as they could, hence the industrial levels of output. Not unlike VCs funding anything and everything for the possibility of the few being true businesses.

[bonoboTP]

The reality is that innovation is hard to plan. It's like outperforming the market. Scientific breakthroughs are about figuring out where are gaps in our knowledge that are fruitful when filled, or where our current understanding is wrong. But if we already knew what we believe wrongly, then we already wouldn't believe it. You can't produce breakthroughs like clockwork and the more thorough work you do, the less opportunity there is to find out later that you were wrong!

The problem is that of course everyone wants the glory of finding out some new groundbreaking innovative disruptive scientific discovery. And so excellence is equated with such discoveries. So everything has to be marketed as such. Nobody wants to accept that science is mostly boring, it keeps the flame alive and passes on the torch to the next generation, but there's far less new disruption than it is pretended. But again, a funding agency wants sexy new finding that look flashy in the press, and bonus points if it supports their political agendas. The more careful and humble an individual scientist is, the less they will seem successful. Constantly second guessing your own hypotheses, playing devil's advocate strongly and doing double and triple checks, more detailed experiments, etc. take longer time and have better chance at discovering that really the sexy effect doesn't exist.

> Makes me wonder, have I turned brilliant or is it quite unimpressive out there?

Obviously, it's impossible to say without seeing their work and your work. But for context, there are on the order of tens of thousands of top-tier AI-related papers appearing each year. The majority of these are not super impressive.

But I also have to say, what may seem "just common sense" may look like that just in hindsight, or you may overlook something if you don't know the related history of methods, or maybe you're glossing over something that someone more experienced in the field would highlight as the main "selling point" of that paper. Also, if common sense works well, but nobody did it before, it's still obviously important to know how well it works quantitatively, including detailed analysis of the details.