[hdjfkfbfbr]

Glad to be of help. Yea that is the repo.

https://replicate.com/blog/run-llama-locally

I found that guide here on hn.

I run it cpu only with 16 threads but yeah perf is good enough.

BTw my 6gb figure is me.measuring from htop so llama2 is likely less.

[Ms-J]

Thanks for the starting point. I'll give an update if I'm able to successfully run the other models. I hope it could help the community.