[gigel82]

The results are pretty good; I wish they'd just publish the models so we can run the inference locally (not too many people have access to 8xA100 to train themselves, though I appreciate including the training data and instructions too).

[menzoic]

Anyone with a few hundred bucks to spare can do it by renting GPUs from a cloud provider. It only cost Stanford $600 to create Alpaca from LLAMA. $100 to generate instructions with GPT-3 and $500 to rent cloud GPUs. The license restriction is due to the use of GPT-3 output to train a model.

[MacsHeadroom]

More like $50 or even $5 or less for the cloud GPUs. Alpaca-7B's compute costs were close to $50 and that was before the 100x cost savings of using LoRA.

A 4bit LoRA fine tune of this project would cost less than $5 to train even up to 30B/33B.

[nickthegreek]

I’d love to see a crowdsourcing platform to donate to specific fine-tuning projects. I would gladly throw some money at someone to do the labor and release the models to the public.

[sahil_chaudhary]

Given how many people are asking for the weights, I'm working to make them available now. So you'll be able to run them locally as well.

[doctoboggan]

If a couple of us get together and throw in some money we could train it on Lambda Labs hardware like the OP suggests. I would volunteer to do it myself but I don’t know enough about training models to guarantee I am not wasting money with a stupid mistake.

[jerrygenser]

Llama may not be licensed for people to share since you need to apply to get one from Facebook for non commercial use. I think it's more of a license issue

Hopefully similar work can be done with LoRA so the fine-tuning is not as expensive

[sahil_chaudhary]

Yeah the license issue is the only reason why I didn't release the weights yet.