|
|
|
|
|
|
by brucethemoose2
929 days ago
|
|
|
> What are the HW requirements for this latest Mistral7B Pretty much anything with ~6-8GB of memory that's not super old. It will run on my 6GB laptop RTX 2060 extremely quickly. It will run on my IGP or Phone with MLC-LLM. It will run fast on a laptop with a small GPU, with the rest offloaded to CPU. Small, CPU only servers are kinda the only questionable thing. It runs, just not very fast, especially with long prompts (which are particularly hard for CPUs). There's also not a lot of support for AI ASICs. |
|
|