[qamononep]

It would be great if you included a section on running with Docker on Linux. The only one that worked out of the box was Ollama, and it had an example. https://github.com/ollama/ollama/blob/main/docs/docker.md

has a docker image but no examples to run it https://github.com/ggerganov/llama.cpp/blob/master/docs/dock...

has a docker image but no examples to run it https://github.com/LostRuins/koboldcpp?tab=readme-ov-file#do...

docker image was broken for me on 7800xt running rhel9 https://github.com/Atinoda/text-generation-webui-docker

[fazkan]

good feedback thanks, would you be able to open an issue

[qamononep]

this repo? https://github.com/AUGMXNT/llm-tracker.info-vault/issues

[lhl]

I think fazkan was confused about which repo you were talking about. For the llm-tracker doc, that's something I maintain. It's based on stuff I test but if you want to submit a PR or issue w/ info in a way that I can verify then I'm happy to add a Docker section.

[fazkan]

haha, I was a bit confused, but I was referring to this one https://github.com/slashml/amd_inference. But the comment applies to other repos as well, do open issues in them, helps the maintainers prioritize features.