Hacker News new | ask | show | jobs
by dr_kiszonka 925 days ago
Do you know if there is a website or spreadsheet where I could enter my RAM, GPU, etc. and see what models I could run locally?
2 comments
SkyMarshal 925 days ago
Yes, https://www.reddit.com/r/LocalLLaMA/. Just ask there and a bunch of Non-Artificial Intelligent agents will give you that answer ;).
link
unstuck3958 925 days ago
While I agree a website/spreadsheet would be convenient, it's not that complicated. As long as GPU is handling 50-75% of the LLM layers, you should get a decent tok/sec speed (unless you're running really really large models).
link
dr_kiszonka 924 days ago
Could you explain to me (in steps) how I would go about calculating how much VRAM I would need to run, say, Mistral 8x7B?
link