Hacker News new | ask | show | jobs
by bluelightning2k 684 days ago
Seems pretty vague. Something to do with half self-hosting open source LLMs with a proprietary docker-like template thing.

Am I on the right track?

2 comments

Not sure I would describe it that way exactly, :). The templates are to get your started. Release itself takes docker compose, docker files, helm, terraform, etc and gives you a way to organize them and a workflow engine to deploy, update and remove every part of the environment. All of the functionality works with the AI templates.

The templates are a good place to start and with our concept of workspaces, you an easily attach data sources and do development with common ai tools and frameworks.

A lot of our customers and others we have talked to don't even know where to start with AI, including RAG. Our templates, infra management and workflow engine make it easy to spin up multiple environments, with varying configs for testing all the way to production.

While a lot of companies are getting into the RAG framework or Fine Tuning framework business, by releasing open source project, it is still very difficult to tie it all together.

Both nvidia and Docker have lamented about this issue to us and this is our attempt to make some of these frameworks easier to use.

Like Together.ai but a bit late to the market I suppose.