[unstuck3958]

While I agree a website/spreadsheet would be convenient, it's not that complicated. As long as GPU is handling 50-75% of the LLM layers, you should get a decent tok/sec speed (unless you're running really really large models).

[dr_kiszonka]

Could you explain to me (in steps) how I would go about calculating how much VRAM I would need to run, say, Mistral 8x7B?