[raybb]

Similarly, is there any half decent general LLM that can run on consumer hardware?

[junhoyeo]

Thinking to add LLaMA or other OSS LLM models soon

[mkl]

Please consider LLaMA.cpp (https://github.com/ggerganov/llama.cpp), which supports a lot of models and doesn't need an expensive GPU.