[jamifsud]

Are there models that can compete with gpt-3.5-turbo on cost per token at scale? From what I'm hearing the 30B+ models net out to a higher $/token but I haven't been able to find anything on the 7B and lower. Thinking about cost specifically here. We're exploring a couple fine tunes for specific tasks we have (we have the data to fine tune with) but gpt-3.5-turbo does reasonably well on the tasks so if the cost is an order of magnitude higher not sure the ROI is there.

[tikkun]

Before considering cost, and you might've already done this, but I'd try and run a 5-shot prompt for your use case with GPT-3.5 and MPT-30B and Falcon-40B. That way you can get a sense of how performance compares, without needing to go through the fine tuning. My guess is that 3.5 might still be significantly better on the 5-shot. I guess you're really comparing base 3.5 with fine tuned MPT-30B/Falcon-40B though, so perhaps for a fairer comparison (until 3.5 fine tuning is available), you could do something like 2-shot with 3.5 and 10-shot with Falcon and MPT.