[PostOnce]

No need to drag a gigabyte of docker stuff into this, just extract the zip file from github and type make into your terminal

congratulations, it now works.

If you're not a developer, maybe you'll have to type sudo apt install build-essential first. Congratulations, now you too, a non-developer, are running it locally.

https://github.com/ggerganov/llama.cpp

[vidarh]

Did that, got a compiler error within seconds within seconds. Looks like it might need a newer version of gcc than is in the distro on my laptop. Which is why people ask for Docker. If it really will work just with make with build-essential on a new enough distro image, a Dockerfile that documents that would be trivial, and does not at all stop people from just typing make if their setup is new enough.