Hacker News new | ask | show | jobs
by brucethemoose2 1153 days ago
You can customize a model with a "cheap" 3090 (or maybe a 7900 XTX), see here: https://github.com/Lightning-AI/lit-llama#finetune-the-model

What OP did is more intense, but Lora/Adapter still can give excellent results.

Pure AI cards for mere mortals aren't really a thing yet.

1 comments

Anyone tried fine tunning on CPU already? I expect it to be much slower, but is it even practical?
Also I bet the pytorch training code is written with CUDA semantics. Maybe a JAX version would work without messing with the code.
Oh no not a chance, finetuning is really compute intense.