Hacker News new | ask | show | jobs
by semessier 310 days ago
still looking for vLLM to support Mac ARM Metal GPUs
1 comments

Yeah. The docs tell you that you should build it yourself, but…
but unlike cuda there's no custom kernels for inference in vllm repo...

I think