|
|
|
|
|
|
by tingletech
10 days ago
|
|
|
it works quite will for me in llama.cpp, but I get more like 20% to 40% speedup on tokens per second. I generally use spec-type = draft-mtp,ngram-mod
spec-draft-n-max = 4
I have not observed any effect on prompt processing, which is usually an order of magnitude faster than generation on my spark. |
|
|