Hacker News new | ask | show | jobs
by properbrew 858 days ago
I've been building upon the LLMWare project - https://github.com/llmware-ai/llmware - for the past 3 months. The ability to run these models locally on standard consumer CPUs, along with the abstraction provided to chop and change between models and different processes is really cool.

I think these SLIM models are the start of something powerful for automating internal business processes and enhancing the use case of LLMs. Still kinda blows my mind that this is all running on my 3900X and also runs on a bog standard Hetzner server with no GPU.
2 comments
tracerbulletx 857 days ago
Don't forget to give some credit to llama.cpp which actually runs the models here and does all the things you're praising it for. This project is more about building a platform on top of it with RAG and function calling.
link
properbrew 857 days ago
Oh yea 100%! llama.cpp and the opensource community in general is truely awesome in getting AI models into the hands of as many people as possible. I think these platforms are the key things to inspire people and get them to see the power of local LLMs in just a few minutes. Can't wait to see what other opensource platforms crop up in 2024 as well.
link
doberst0103 857 days ago
Absolutely! We credit Georgi Gerganov and llama.cpp for the amazing advancement in quite a few of our YT videos. He is truly a hero.
link
doberst0103 857 days ago
Thank you so much for the awesome feedback!
link