[conshama]

Somehow I never got the hype around Groq. They served models with fast inference speed - that sounded great in theory - and as a user I was looking forward to use them. But, when I did try, I discovered that they are quantizing the models underneath. And they dont even disclose it. So I stopped using them.

The whole thing never made any sense to me - but I guess AI hype is a thing.

[adityashankar]

I believe despite quantisation they were still extremely fast, which is still incredibly useful if you don't need high precision/accuracy (which is good enough for many use cases)