[unreal37]

Someone was able to replicate GPT 3.5 with $500. The training of models is getting very cheap.

[1] https://newatlas.com/technology/stanford-alpaca-cheap-gpt/

[ftufek]

I've tried it, sure it's good, but not even close to the real thing. But yes it's getting cheaper through better hardware, better data and better architectures. Also it builds on Facebook's models that were trained for months on thousands of A100 GPUs.

[lern_too_spel]

By fine-tuning a leaked model trained with a lot more money than that. If somebody leaks the GPT 3.5 model, can I say that I replicated it for $0?