[ilaksh]

I don't know if they mention Code Llama but that has been a huge milestone. But from what I've heard it's still at least 15% behind GPT-4 for code. But open models are gaining ground.

Although ChatGPT Enterprise kind of changes the benchmark again in away.

[Havoc]

>But from what I've heard it's still at least 15% behind GPT-4 for code

More. But you're comparing the performance of something that runs on a macbook vs something out of a datacenter full of H100s.

[ChatGTP]

I guess the idea here is that it's probably not wise to blindly invest $10BUSD on something which might be ubiquitous and free in 6 months.