| Hey! Kudos to them firstly! Its the same problem statement but different solutions. They seem to use tracing and tracing generally adds a lot of overhead on the application. Our method tries to avoid that plus added latency is ~0.01s in our case. Tracing seems really useful wen using RAG based approach so we are working on adding it as an option too but for simple fine tuned approach, We believe Doku should do a good job. We prioritise self-hosting a lot more. Setting up Doku is very very simple(2 Doku components and ClickHouse) for both Docker and we do have a Helm chart for Kubernetes which I generally found a bit tough in the other tools + self-hosting IMO is easier as data regulations are not a big headache. We also allow you to add multiple clickHouse (where we store the LLM Monitoring data) to Doku so that you can easily separate staging and production data. If you are already using ClickHouse for any other purposes, You can easily connect that too! and as you said our connections, IMO everyone is using some sort of an observability tool, We don't want users to learn and educate to use Doku UI in those cases, Just simple connect and use your existing observability platform (So if you dont wanna use Doku UI, thats one less component you need to run) rest I think is the difference in what LLMs we can monitor but thats not big as most users Ive talked to right now are still at a stage where they are experimenting somehow with OpenAI Would love for you to try it out and understand the things you feel we should add! Thanks! |