[rafram]

> Haven’t seen much of that from Microsoft

Microsoft is the de facto controlling shareholder in OpenAI. They provide all the money, compute, and backing, and have full access to the models. If OpenAI collapsed tomorrow, Microsoft would absorb its key employees (as they almost did during the board debacle) and everything would continue under the Microsoft umbrella. “OpenAI” is just a shinier name for work that is being done under the near-total control of Microsoft.

[aik]

The money and compute is not the innovation. The LLM models and associated tools are, which is work by OpenAI employees and teams, not Microsoft employees/teams.

[danjl]

In that vein, I'd say that most of the LLM research was done at Google. OpenAI productized faster.

[aik]

Confused here, is your argument here that OpenAI is not responsible for any innovation when it comes to LLM tech today? I’m curious about why you so strongly want to believe that?

Nobody knew that scaling transformer architecture would lead to the emergent intelligence we see today. Among other things, OpenAI did R&D for years on that. Also the only situation where this could true is if Google knew that LLMs could lead to this intelligence and decided to not make it happen, (along with every other tech company now that is furiously trying to catch up to OpenAI), which is absurd.

[danjl]

You seem to be inflating the emotional importance of my comment. Google did an enormous amount of the research prior to scaling. I merely pointing out that if there's credit to be given out, a bunch of it goes to Google.

[aik]

Agreed.

[HarHarVeryFunny]

Actually Google were building and scaling transformers at same time as OpenAI - BERT (following Allen AI's ELMo), T5, Meena, LaMDA (chatbot - preceding ChatGPT by a year or two), PaLM ...

It seems that Google really didn't know what to do with the tech, and hadn't figured out a way to control it (OpenAI's RLHF - critical for ChatGPT's success). It's a bit ironic that DeepMind were doing so much with RL, but Google Brain being separate at the time apparently were not consulting with them or tapping into their expertise.

[Dylan16807]

> not responsible for any

That's a very strange way to read (the inverse of) the word "most".

[rafram]

It might’ve been easier to hire that talent under the shiny OpenAI umbrella, but as I said, Microsoft could absorb the entire thing overnight if it wanted to. And pay them enough to make them stay.

[ripjaygn]

> Microsoft is the de facto controlling shareholder in OpenAI

No, they are not even a shareholder.

> Microsoft is entitled to up to 49 percent of the for-profit arm of OpenAI's profits, according to reports. But that's not the same as 49% ownership. That investment does not result in Microsoft owning part of OpenAI

[rafram]

https://www.bloomberg.com/opinion/articles/2023-11-20/who-co...

Note that that diagram is now a little out of date - Microsoft now has an actual (albeit non-voting) board seat, not just effective control.