[talldayo]

> It creates barriers only they can afford, limiting competition and diversity.

I would argue this is inherent to the training and compute cost of all large language models.

[binary132]

Quantized Llama 3.1 can run on an Amazon GPU instance for $32/hr now

[talldayo]

That's still prohibitively expensive for anyone that doesn't intend to make their money back. For training it's even more outrageous.

[binary132]

$64K isn’t even one developer’s salary.

[talldayo]

And how many people can afford to pay a developer's salary after utilities and mortgage?

[binary132]

I think you’re missing my point: hiring someone is not that costly relative to training a billion-dollar model from scratch, so the barrier is definitely getting lower. An individual proprietor with a moderately successful business (or just business loans, let’s face it) can hire at least a couple of employees.

[anon373839]

Small models can be fine-tuned to perform specific tasks with similar accuracy to large models. Small models can be served for internal use with a VERY modest hardware outlay.

There are also providers now that will let you upload low-rank adapters you have trained in top of open foundational models, so that you can use their efficient serverless infrastructure with your fine-tuned models. This requires even less capital.

None of this would exist had OpenAI’s vision of centralized, locked-down API access become the reality.