[biophysboy]

It is a real shame too, because industry is completely incapable of doing basic research. Universities make the fuzzy ideas, and companies turn them into widgets. The only exceptions in history to this are the monopolies, which have their own obvious problems. They cannot produce non-rival, non-excludable goods - stuff that's hard to patent.

[onetimeusename]

Sometimes. I've seen researchers who just churn out useless junk for citation mining and I don't see a lot of overlap between their work and what industry does. That's probably one of the most demoralizing things about academia in my opinion. You sometimes have to be obsequious to people whose goal is just citation farming and whose papers are useless junk filled with buzzwords. I see this a lot in systems and security research. But I also know some researchers who do amazing work and whose research directly gets used in industry.

[biophysboy]

Yes, I hear you on how academia chases metrics. I would argue this phenomena is not worse than Company Z making a boilerplate AI chat tool that is no more useful than the flagship popular products. I think the fairest comparison is comparing the best researchers in academia/industry. I think they accomplish different things because they have different goals/incentives.

[ricksunny]

Vaswani, A., et al. (2017) Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, 4-9 December 2017, 6000-6010.

Generally understood to be an output of Googlers.

[biophysboy]

Transformers are an applied science: https://patents.google.com/patent/US10740433B2/en

Basic research would be something like optimal control theory, which came well before the transformer design.

I'm not trying to be evasive; I can see how my distinction could be seen as conveniently just outside industry's purview. Put it this way: I think companies, particularly small ones, are incentivized to pursue well-known methods/materials. Innovation modulates and optimizes.

[daedrdev]

There is plenty of research, it is all kept as trade secrets because they are in a competitive business environment