[version_five]

What kind of deep learning startups survived or thrived? The underlying technology (of gpt et al) is racing towards being a commodity. There are very few "pure play" "AI" companies left from the deep learning wave, and what few there are pivoted to become SaaS with some ML features. Something similar will happen here. Companies that solve a business pain point, irrespective of whatever "AI" will get somewhere.

[ignite]

I wonder if the issues around owning of training data may allow larger, well funded companies to acquire ownership (for example, buying stack overflow content). This might allow larger players to starve smaller players.

[version_five]

If we're not already, I'd expect to see lots of previously public content get locked down so it can be monetized. It will be interesting to see what the long run equilibrium looks like, as in will large players play ball?

I'd guess that the future high value application will still have lots of fine tuning on more specialized datasets, which will continue to be a bottleneck and be considered valuable, while the large scale training data will be less important (with respect to transformer llms, who knows if there is some newer breakthrough). Same way that e.g. everybody uses CNNs pertained on imagenet, and fine tunes on application specific stuff, and there was not much of a commercial push (imo) to rush out and build a better general purpose large scale image set like imagenet.