Hacker News new | ask | show | jobs
by root_axis 52 days ago
Not really. The hardware requirements remain indefinitely out of reach.

Yes, it's possible to run tiny quantized models, but you're working with extremely small context windows and tons of hallucinations. It's fun to play with them, but they're not at all practical.
1 comments
ac29 52 days ago
The memory requirements aren't that intense. You can run useful (not frontier) models on a $2-5K machine at reasonable speeds. The capabilities of Qwen3.6 27B or 35B-A3B are dramatically better than what was available even a few months ago.

Practical? Maybe not (unless you highly value privacy) because you can get better models and better performance with cheap API access or even cheaper subscriptions. As you said, this may indefinitely be the case.

link
root_axis 52 days ago
> The capabilities of Qwen3.6 27B or 35B-A3B are dramatically better than what was available even a few months ago.

Yes, a lot better, but still terribly unreliable and far less capable than the big unquantized models.

link