[bhouston]

13B models probably cannot directly compare with ChatGPT 4 which maybe +1T parameters or a 5 way MoE of 200B each - or something like that. So you can not likely run a model competitive with ChatGPT locally in the near term.

[tombert]

I have a server with a bunch of PCIe slots and like 4 Nvidia GPUs with 24GB of RAM each. What's the best model I can realistically run?

[bhouston]

Here are some scorecards:

https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...

https://paperswithcode.com/sota/sentence-completion-on-hella...

https://paperswithcode.com/sota/common-sense-reasoning-on-wi...

https://paperswithcode.com/sota/common-sense-reasoning-on-ar...

https://paperswithcode.com/sota/common-sense-reasoning-on-co...