Hacker News new | ask | show | jobs
by latchkey 16 days ago
Not sure I understand. If they support MI300x, their self-hosted will run on our hardware.
1 comments
erichocean 9 days ago
If it was that easy, I wouldn't have commented.

It's not, which is why it would be nice if they did the actual work (on your hardware).

I would 100% pay $16/hr to run a self-hosted instance, but I won't spend thousands of dollars to (maybe) get it working (my time + the hardware).

link
latchkey 9 days ago
Ok, sure. Valid. Have you asked them to support V4?

https://docs.modular.com/max/models/

I agree with you though, serving up inference is secret sauce for a lot of teams and not everyone publishes how to do it because of the costs involved in doing so. They need an ROI.

link