[orenlindsey]

That's really fast. But it mostly seems to be because they made a custom chip. I want to see an LLM that is so highly optimized that it runs at this speed on more normal hardware.

[stavros]

But the point is that they made a custom chip. I want to see buy their custom chip so I can have an "LLM box" in my house.

I'd pay quite a bit of money to have a Mixtral box at home, then we'd all have our own, local assistant/helper/partner/whatever. Basically, the plot of the movie Her.

[agildehaus]

That'd be nice, but we could also just make this hardware normal.