|
|
|
|
|
|
by l3jin
975 days ago
|
|
|
Universal deployment is indeed attractive. I have tested the Llama2-70B on 7900 XTX. Love the performance! Also saw a report earlier today on MLC’s discord about AMD MI-100: GPU Count | Model Size | Prefill Speed | Decode Speed 1 | 33b | 102.2 | 22.3 2 | 33b | 112.3 | 33.0 4 | 33b | 144.8 | 41.2 |
|
|