Neat, eventually would like to run a purely local LLM Emacs shell https://github.com/xenodium/chatgpt-shell. For now ChatGPT only, but working on making more generic/reusable.
The oobabooga setup feels a lot more mature and has a larger community. Skimming OP's repo, there seems like a lot of fiddling and faffing with JSON to get things running.
Do you know what is the way to find models compatible with oobabooga/text-generation-webui ? I downloaded one with the included script and that worked, but if I try different ones it seems there are so many formats so no idea say how do I search huggingface or google to find the correct format.
I would like to try this new quantized LLAMA versions with the GUI, I can
run them in the CLI on the CPU but llama.cpp uses ggml formats .