[shortrounddev2]

How you all used these things for anything useful? I can't get them to give useful results on my 3060 8gb. If I wanted to get decent results I think I'd need to rent a GPU node somewhere, but chatGPT is still free

[SushiHippie]

The 4bit quantized 13B models, give really decent answers (not as good as gpt4, but often as good as gpt 3)

[nmfisher]

I know it might be asking a lot, but it would be great if someone could put up a HF space so I could try all the various flavours/sizes.

[lazylion2]

/r/LocalLLaMA/

[nmfisher]

I'm already subscribed (and I already ran the small version locally), but I'd still like to be able to quickly evaluate the models online in a couple of minutes, rather than going through the rigmarole of downloading & running every new model/variant locally.