[jakozaur]

There is an effort in OpenTelemetry to create a standard query language for observability. There were a lot of discussions with a lot of opinions; there were even several talks during KubeConEU about that:

https://sched.co/1tcyx

https://sched.co/1txI1

We are still waiting for a compelling implementation that will show the way.

[ople]

Why not just use SQL? With LLMs evolving to do sophisticated text-to-SQL, the case for a custom language for the sake of simplicity is diminishing.

I think that expressiveness, performance and level of fluency by base language models (i.e. the amount of examples in training set) are the key differentiators for query languages in the future. SQL ticks all those boxes.

[jakozaur]

You are right. SQL is the best language, but it likely needs some extensions. See SQL with pipe syntax. Read Google paper or try it out in Big Query.

There are a lot of fundamentals in observability, but there are very verbose in SQL:

- rate operator, which translates absolute value to rate, possible with SQL and window functions, but takes many lines of code

- pivot, where you like to see the top 5 counts of errors of most hit-by-error microservices plus others over time

- sampling is frequent in observability and will be useful for LLMs, it is a one-liner in SQL with pipe syntax, even customizing specific

I actually believe LLM gen AI plays extremely well with pipe syntax. It allows us to feed partial results to LLM, sampling as well as show how LLM is evolving queries over time. SQL troubleshooting is not a single query but a series of them.

Still, SQL with pipe syntax is just syntactical sugar on SQL. It let's you use all SQL features as well as compiles to SQL.