Well. Right now buying hardware to run your own models tops off at about 32gb VRAM at any price point that's not insane. Sure you can get a Mac mini, or a PC equivalent. But the problem is RAM.
More RAM means bigger models, which means smarter models.
Which is why Qwen and Gemma have been so interesting to a lot of us who run our own... Now 32gb VRAM isn't so bad, as these models can be run on that with decent results.
Where this gets interesting is in a couple years, when all the A100, etc, all the Enterprise hardware hits eBay.
More RAM means bigger models, which means smarter models.
Which is why Qwen and Gemma have been so interesting to a lot of us who run our own... Now 32gb VRAM isn't so bad, as these models can be run on that with decent results.
Where this gets interesting is in a couple years, when all the A100, etc, all the Enterprise hardware hits eBay.