[roskilli]

There's a lot of interest in this space with respect to analytics on top of monitoring and observability data.

Anyone interested in this topic might want to check out an issue thread on the Thanos GitHub project. I would love to see M3, Thanos, Cortex and other Prometheus long term storage solutions all be able to benefit from a project in this space that could dynamically pull back data from any form of Prometheus long term storage using the Prometheus Remote Read protocol: https://github.com/thanos-io/thanos/issues/2682

Spark and Presto both support predicate push down to a data layer, which can be a Prometheus long term metrics store, and are able to perform queries on arbitrary sets of data.

Spark is also super useful for ETLing data into a warehouse (such as HDFS or other backends, i.e. see the BigQuery connector for Spark[1] that could write a query from say a Prometheus long term store metrics and export it into BigQuery for further querying).

[1]: https://cloud.google.com/dataproc/docs/tutorials/bigquery-co...

[tixocloud]

Thanks for sharing. It’s interesting to see the space gaining steam. What sort of things are people looking at?