Local LLMs are defo the way forward for governments and certain private companies, that was a huge blocker with making these types of products.
Haven't used ollama but I've used GPT4ALL and that has the ability to turn on a simple rest api that mimics the standard openai completions endpoint, so would be super easy!
Haven't used ollama but I've used GPT4ALL and that has the ability to turn on a simple rest api that mimics the standard openai completions endpoint, so would be super easy!