Microsoft and OpenAI, Google losing $100bn from a bad demo, layoffs, cost cutting pressure to innovate again... I feel the days of full openness in AI research from corporations are over.
I wouldn't be remotely so quick to throw in the towel. ML research tends to operate in "jumps" and plateaus. At this point, the concepts behind the big LLMs are relatively well known and the bottleneck is cost of compute + cost of training data. Thing is, cost of compute keeps coming down.
OpenAI's "win" wasn't even so much in the research but in the design of ChatGPT as an interface. Its own model makes the same kinds of egregious mistakes as google and FB's own LLMs. Also, OpenAI was willing to just deal with the ethical fallout of releasing it into the wild with the ability to generate authoritative sounding falsehoods.
I suspect we're going to go back to a period soon where a lot of the innovation we're seeing is around interfaces and infra to make interacting with LLMs natural and applying them to product use cases where they make sense.
I really don't blame OpenAI for ethical issues over opening up access to ChatGPT. They're not claiming it's responses are factually correct, and arguably by making it openly available they have done more than anyone else to raise awareness of the risks and limitations of LLMs. We need access to these things to make informed decisions of what are or are not appropriate uses.
Microsoft and Google are a different story, they're specifically pushing these as authoritative sources of information. If we hadn't had access to ChatGPT and the ability to learn it's ins and outs, it might have taken longer to expose so may of the flaws in the Microsoft and Google services.
I think we indeed hit "peak open-source" for AI and there unfortunately won't be as much sharing in the coming years. When the economy is down, people and companies think more in "zero-sum-game". I hope to be proven wrong.
The golden age of open-source AI is ahead of us. Open-source AI companies are being launched and funded. High quality, large, labeled data sets have never been more accessible, and scaling law plateaus means there is going to be a lot more momentum on data- and compute-optimization, meaning current SOTA models will start fitting on smaller and smaller hardware, down to commodity hardware.
This. We already had discussions of 'attacks' on models based on public data sets so 'good sets' may soon become the thing to go after ( and suddenly data brokers may really want to up the prices of their sets ). We might actually see more privacy as a result as the data brokers will start charging a premium for clear sets.
Naturally, as predictions go, don't quote me on that. I was wrong before.
Yeah I don't expect that to last very long. Once it is discovered you can have an AI army that will tear apart any human force like tissue paper, AI will be classified and regulated as a WMD. They'll amend the Constitution if they have to, or get SCOTUS to do it for them.
This is why activist judiciaries inventing rights that plainly are not there (and in fact, supporters at the time understood that this was on shaky ground legally) are dangerous to rely on.
This is what everyone needs to watch. It was Stability.ai that spooked OpenAI for disrupting DALLE-2 with a open-source AI model; Stable Diffusion. I am betting they are going to do it again for ChatGPT.
The endgame for this AI race is obvious and when it comes to AI models, open-source ones always disrupt fully closed source AI companies.
But first, we'll see which 'AI companies' will survive the lawsuits, regulations and fierce competition for funding.
OpenAI's "win" wasn't even so much in the research but in the design of ChatGPT as an interface. Its own model makes the same kinds of egregious mistakes as google and FB's own LLMs. Also, OpenAI was willing to just deal with the ethical fallout of releasing it into the wild with the ability to generate authoritative sounding falsehoods.
I suspect we're going to go back to a period soon where a lot of the innovation we're seeing is around interfaces and infra to make interacting with LLMs natural and applying them to product use cases where they make sense.