[ftufek]

It's easier for large rich companies with infrastructure and datasets. It's very hard for small startups to build useful real world models from scratch, so you see most people building on top of SD and APIs, but that limits what you can build, for example it's very hard to build realistic photo editing on top of stable diffusion.

[adam_arthur]

Most of the cutting edge models are coming from companies with a few dozen to a few hundred people. Stability AI is one example.

Training an AI model, while expensive, is vastly cheaper than most large scale products.

This wave will be nothing like the SaaS wave. Hyper competitive rather than weakly-competitive/margin preserving

[ftufek]

I wrote it from the perspective of a small startup (<10 people, bootstrapped or small funding). I think it's far cheaper and easier to build a nice competitive mobile app/saas than to build a really useful model.

But yes I agree, it will be very competitive with much smaller margins.

[unreal37]

Someone was able to replicate GPT 3.5 with $500. The training of models is getting very cheap.

[1] https://newatlas.com/technology/stanford-alpaca-cheap-gpt/

[ftufek]

I've tried it, sure it's good, but not even close to the real thing. But yes it's getting cheaper through better hardware, better data and better architectures. Also it builds on Facebook's models that were trained for months on thousands of A100 GPUs.

[lern_too_spel]

By fine-tuning a leaked model trained with a lot more money than that. If somebody leaks the GPT 3.5 model, can I say that I replicated it for $0?