Hacker News new | ask | show | jobs
by lobster45 22 days ago
What type of hardware, GPU are you utilizing? I have the same model self hosted at home
1 comments
yolo-auto 22 days ago
1x MI300x , rocm/sglang , highly optimized image, FP8, FP16 Cache (this fixes looping issues with qwen3.6-35b

~100tok/s average

link