[tombert]

Is that true? I was running Llamas on my laptop a few days ago, and it was giving measurably worse results than ChatGPT. I think it was the uncensored 13B model, but if you got something that's on par with ChatGPT that I can run on my own hardware I'm pretty interested.

[bhouston]

13B models probably cannot directly compare with ChatGPT 4 which maybe +1T parameters or a 5 way MoE of 200B each - or something like that. So you can not likely run a model competitive with ChatGPT locally in the near term.

[tombert]

I have a server with a bunch of PCIe slots and like 4 Nvidia GPUs with 24GB of RAM each. What's the best model I can realistically run?

[bhouston]

Here are some scorecards:

https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...

https://paperswithcode.com/sota/sentence-completion-on-hella...

https://paperswithcode.com/sota/common-sense-reasoning-on-wi...

https://paperswithcode.com/sota/common-sense-reasoning-on-ar...

https://paperswithcode.com/sota/common-sense-reasoning-on-co...