[stefs]

well, we can AI inference on our desktops for $500 today, just with smaller models and far slower.

[ryao]

There is no need to use smaller models. You can run the biggest models such as llama 3.1 405B on a fairly low end desktop today:

https://github.com/lyogavin/airllm

However, it will be far slower as you said.