[sometimelurker]

llama.cpp + llama-serve, I use it. if you like terminal, pi