Hacker News new | ask | show | jobs
by freeqaz 822 days ago
Dang, so is Groq the cheapest for running Mixtral then? How capable is that model versus GPT 3.5 for basic tasks?
1 comments

It's not too far behind. But Groq is currently not billing and I think it's because they haven't finished programming their billing system. So with such a fast service being free, there is a lot of queueing and presently is unusable because of random response times. Once it finally processes the request it's quite fast but there is a line.

So I am going with Claude 3 Haiku. Hopefully Groq will be able to start charging soon and the load will be more tolerable to service without queues.

Is there a production case you are using Haiku for? If so, can you outline it?