|
|
|
|
|
|
by marci
929 days ago
|
|
|
If I understand correctly: RAM Wise, you can easily run a 70b with 128GB, 8x7B is obviously less than that. Compute wise, I suppose it would be a bit slower than running a 13b. edit: "actually", I think it might be faster than a 13b. 8 random 7b ~= 115GB, Mixtral is under 90. I will have to wait for more info/understanding. |
|
|