|
|
|
|
|
|
by patcher99
775 days ago
|
|
|
We do Otel metrics for the main things needed in dashboarding
1. Requests (Counter)
2. Tokens (Counter) (Seperate metric for Prompt Tokens, Completion and Total Tokens)
3. Cost (Histogram) I did attach a Grafana dashboard too (Works for Grafana Cloud, Ill get something for OSS this week)
https://docs.openlit.io/latest/connections/grafanacloud Since Otel doesn't yet support Synchronous gauge + Users wanted a trace of the LLM RAG application, We opted to Traces which is quite standard now in LLM Observability tool. Lemme know if you had something more in mind, Love getting feedback from amazing folks like you! |
|
|