[kouteiheika]

Sure, I'm just comparing the baseline costs of finetuning. Assuming you own the hardware and optimize the training I'm guessing you could easily get the costs significantly lower than $0.1/M tokens (considering I can get the $0.1/M right now using publicly rented GPUs, and whoever I'm renting the GPU from is still making money on me), and if you're only doing LoRA that cost would go down even further (don't have the numbers on hand because I never do LoRA finetuning, so I have no idea how much faster that is per token compared to full finetuning).

So your $2/M tokens for LoRA finetuning tells me that you either have a very (per dollar) inefficient finetuning pipeline (e.g. renting expensive GPUs from AWS) and need such a high price to make any money, or that you're charging ~20x~30x more than it costs you. If it's the latter - fair enough, some people will pay a premium for all of the extra features! If it's the former - you might want to consider optimizing your pipeline to bring those costs down. (: