[GaggiX]

People should not worry about Stack Overflow or Microsoft, etc., but about individuals who can train from scratch and fine-tune the models themselves, renting or bidding powerful GPUs is not particularly expensive, downloading millions of images is not expensive, curating the dataset is not expensive, and it is only because of these people that it is worth using models like SD. And as for the article, these individuals are much more difficult to put pressure on.

[mmiliauskas]

What do you mean "not particularly expensive", GPT-3 I think costs like $4M to train. And that is just the pre-training part.

[GaggiX]

I'm obviously not talking about training a 173B model from scratch ahah

If you want to train a model from scratch I was talking more about a small 64x64 DDIM models (that you can then upscale with openly available upscaler DDIM models and maybe finetune). In most cases however it's better to just finetune the models available, the point is that pressuring companies to not train a model on the data they host doesn't really do much if a single individual can scrape the entire DeviantArt site and finetune a model on it.

[freehorse]

But why should they be afraid of these models? These models will not gonna be competitive enough compared to the models of the larger companies, and considering that training cost > inference cost, it will never be competitive in market sense to small-scale train different models from scratch. So these models are never gonna reach the impact of the larger ones in the end.

[GaggiX]

By finetuning a model (correctly) you will retain its capability while shifting the distribution to what you desire, as I have said before "it is only because of these people that it is worth using models like SD".