[brucethemoose2]

You can customize a model with a "cheap" 3090 (or maybe a 7900 XTX), see here: https://github.com/Lightning-AI/lit-llama#finetune-the-model

What OP did is more intense, but Lora/Adapter still can give excellent results.

Pure AI cards for mere mortals aren't really a thing yet.

[theolivenbaum]

Anyone tried fine tunning on CPU already? I expect it to be much slower, but is it even practical?

[brucethemoose2]

Also I bet the pytorch training code is written with CUDA semantics. Maybe a JAX version would work without messing with the code.

[brucethemoose2]

Oh no not a chance, finetuning is really compute intense.