Big Tech hasnt dominated anything. They got trounced by a 400 person startup. Alexa, Google Search, FB research all spend untold money on ML systems. Neither is releasing game changing products anywhere close to OpenAI.
The Transformer architecture that powers GPT models is based on a now-famous Google Brain paper, Attention is All You Need. You're right that OpenAI has productized this research much faster than Big Tech, but the research itself came from established R&D in Academia and Big Tech.
The diffusion papers most analogous to the transformer paper are from Berkeley, Stanford, and Freiburg. Imagen wasn’t foundational in the way you’re suggesting.
OpenAI was backed by probably the biggest tech company around, Microsoft. Without the computing afforded by Azure, they would have problems training and running the LLM.
It is true that big companies have brands and existing revenue stream to protect (Google), so maybe only a little company with nothing to lose could do what OpenAI did.
And maybe MS found the perfect arrangement giving them an ability to adjust how closely to associate themself with the tech according to need.
But is this all necessarily a good thing? OpenAI gave MS access to GPT and MS clearly wants to use GPT to destroy Google, so Google now has respond aggressively, and now we have exactly the AI arms race that many predicted would lead to the worst outcomes.
I disagree. All of the big players have been involved in some way, creating their own systems (which are quite competitive) as well as laying much of the foundation for existing models with research they published.
I wouldn't say big tech was "trounced" by OpenAI. Advances in LLMs are driven by the research community. OpenAI is a small, well-resourced organization with the sole purpose of quickly deploying and hyping state of the art AI models. Big tech naturally has a slower product development cycle - and will make full use of LLMs in due time.
> I wouldn't say big tech was "trounced" by OpenAI.
I mean not a single “big tech” company is offering any LLM products not powered by Open AI. It’s been almost 3 years since GPT-3 was released. I’m not sure they can catch up at this point. And I think the GPs point was a good one: big tech threw billions and billions at personal assistants. Nothing innovative was delivered.
Probably, but using a transformer isn’t Open AI’s advantage. A widely available language model with “emergent abilities” is Open AI’s advantage. The fact that it’s a transformer is more of an implementation detail.
Yes and tech was so path breaking and innovative that the company sold its 49% share for 10 billion dollar /s. Sam Altman is extremely experienced in the VC world and could have easily raised billions for potential value. He wouldn't have sold half of his company within 6 months of releasing chatGPT if there was any thing remotely defensible in the product