| You have two requirements that are at odds: * Macbook is not an option * Want to be able to mess around with some local LLMs. Your choices for a Window laptop that can run a local LLM is either to get a large amount of system RAM and have it be abysmally slow, or to run a very tiny model on a discrete GPU which will (a) not be very good due to its small size and high quantization and (b) evaporate your battery life. If you want to run local LLMs on a laptop and actually have them be useful, a Mac is currently the only real choice. That said, with the money you save buying a Linux laptop instead, you can pay for a lot of tokens for whatever hosted LLM you want and it will be higher quality than what you could potentially run locally on a Mac. |
I've not tried local LLMs on Windows, but I do loads with 'em on a three-year-old Legion running Arch.
That said, whilst small local models are nice for some use-cases, I'm leaning more towards APIs these days. I like the better selection of models and the ability to use them without bringing my machine to a halt.