Hacker News new | ask | show | jobs
by arwineap 778 days ago
On the span is an interesting attribute to view, but had you considered using otel meters/metrics for that instead?

I think cardinality in span attributes can be a problem, and meters are better for aggregating and graphing
1 comments
patcher99 778 days ago
We do Otel metrics for the main things needed in dashboarding 1. Requests (Counter) 2. Tokens (Counter) (Seperate metric for Prompt Tokens, Completion and Total Tokens) 3. Cost (Histogram)

I did attach a Grafana dashboard too (Works for Grafana Cloud, Ill get something for OSS this week) https://docs.openlit.io/latest/connections/grafanacloud

Since Otel doesn't yet support Synchronous gauge + Users wanted a trace of the LLM RAG application, We opted to Traces which is quite standard now in LLM Observability tool.

Lemme know if you had something more in mind, Love getting feedback from amazing folks like you!

link