|
|
|
|
|
|
by segmondy
778 days ago
|
|
|
8B, and it got better this morning, they merged in flash attention so I can now load almost 500k tokens with (96gb of vram) With that said, you can possibly have this kind of resource, this is a cheap build. Mixture of old and used GPUs. |
|
|