Show HN: Open-Source LLM Observability and Export to Grafana, Datadog etc.

[patcher99]

Hi Everyone! This is the maintainer of Doku, an open-source LLM Observability tool that you can self-host and run easily (its 3 simple components)

Doku currently can monitor OpenAI, Mistral, Anthropic, Cohere and Azure OpenAI usage like Performance, Costs, Tokens and user interactions. This is a starting point for us, So let us know what you'd like to see in LLM Monitoring.

Also, We fully understand that you are already using some observability tool, So we did build our connections with these which allow auto-export to your Observability tool. Currently Doku can export to Grafana Cloud, DataDog, New Relic, Dynatrace and SigNoz. Too make you job easier, We have build dashboards that you can easily import into all of these. Checkout -> https://docs.dokulabs.com/latest/connections/intro

Doku is self-hosted and Open-source at heart. we’d love for you to try it out and share what you think. Check it out at https://docs.dokulabs.com/

[adrmtu]

Congrats on the Show! How’s this different from https://github.com/langfuse/langfuse? The exports seems really interesting

[patcher99]

Hey! Kudos to them firstly! Its the same problem statement but different solutions.

They seem to use tracing and tracing generally adds a lot of overhead on the application. Our method tries to avoid that plus added latency is ~0.01s in our case. Tracing seems really useful wen using RAG based approach so we are working on adding it as an option too but for simple fine tuned approach, We believe Doku should do a good job.

We prioritise self-hosting a lot more. Setting up Doku is very very simple(2 Doku components and ClickHouse) for both Docker and we do have a Helm chart for Kubernetes which I generally found a bit tough in the other tools + self-hosting IMO is easier as data regulations are not a big headache.

We also allow you to add multiple clickHouse (where we store the LLM Monitoring data) to Doku so that you can easily separate staging and production data. If you are already using ClickHouse for any other purposes, You can easily connect that too!

and as you said our connections, IMO everyone is using some sort of an observability tool, We don't want users to learn and educate to use Doku UI in those cases, Just simple connect and use your existing observability platform (So if you dont wanna use Doku UI, thats one less component you need to run)

rest I think is the difference in what LLMs we can monitor but thats not big as most users Ive talked to right now are still at a stage where they are experimenting somehow with OpenAI

Would love for you to try it out and understand the things you feel we should add!

Thanks!