|
|
|
|
|
|
by Lwerewolf
8 days ago
|
|
|
...they'd messed something up in that quant, apparently fixed promptly. ds4 now supports it as well and it's... well, context-limited (<=250k on a 128gb machine) but it positively flies on an m5 max - the 60tok/s decode / ~500tok/s prefill is real. |
|
|