|
|
|
|
|
|
by ekojs
459 days ago
|
|
|
> The bottleneck then becomes how to self-host the finetuned model in a way that's cost-effective and scalable It's not actually that expensive and hard. For narrow usecases, you can produce 4-bit quantized fine-tunes that perform as well as the full model. Hosting the 4-bit quantized version can be done on relatively low cost. You can use A40 or RTX 3090 on Runpod for ~$300/month. |
|
|