[attentive]

and Groq does 485.08 T/s on mixtral 8x7B-32k

I am not sure local models have any future other than POC/research. Depends on the cost of course.

[tome]

(Groqster here) For anyone who wants to try it, you can go to https://chat.groq.com/ and choose Mixtral from the drop-down menu. Also, feel free to ask me any questions about Groq hardware or service.