|
|
|
|
|
|
by embedding-shape
33 days ago
|
|
|
Comparison with a RTX Pro 6000, with DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf: prefill: 121.76 t/s, generation: 47.85 t/s Main target seems to be Apple's Metal, so makes sense. Might be fun to see how fast one could make it go though :) The model seems really good too, even though it's in IQ2. |
|
|