[jhbadger]

>Even if the subsidies that drive increasingly sophisticated models evaporate, we'll still have the existing models we can run on commodity hardware.

Exactly. So much discussion in the media and elsewhere about generative AI assume that AI-as-a-service from big companies like OpenAI is the only way forward, and if they die, so does AI in general. But we already can run quite powerful models locally. For example, I don't think Cory's example of a $10/month service to draw D&D character portraits makes sense now. Surely anyone geeky enough to play D&D could download and install InvokeAI (or other similar open-source program) and create their portraits for free.

[__loam]

The inherent problem I'm seeing is that training the base models is incredibly expensive and computationally intensive. You can't do it without institutional capital. Gpt-4 I believe cost 100 million to train. I'd bet they're also losing money on a lot of customers because inference isn't that cheap either. If investors realize something like investing in this stuff isn't paying off, it becomes pretty hard to make new models. Maybe someone would have to develop a distributed system like folding at home to get it done.

[jhbadger]

Well, the various models on sites like civitai are definitely not created by institutions -- especially not the many, many NSFW ones. Yes, you can argue that they are relying on existing models from StableAI and the like, but even if those were never updated, that wouldn't stop the creation of new models on top of them. We kind of see this already; most checkpoint models and loras on civitai are still based on the older Stable Diffusion 1.5 rather than the newer Stable Diffusion XL.

[__loam]

I mean that's kind of my point. Derivative models are all dependent on models built by institutional players like stability. Is there a limit to how good civitai models can be? If there is, you can't move past that without higher quality base models, which requires a lot of capital. That's just what my intuition says.