Prometheus cofounder here - we're happy to take any questions. Huge congrats to everyone who made this release possible and for all the excellent work over the years that lead up to this!
Any plans on more native support for Kubernetes? The relabeling spaghetti config you end up with is very confusing and unreadable.
Granted, you don't need to touch this confirmation very often, but anyone who's going to operate the cluster will need to understand it thoroughly.
The fact that it's ad-hoc (prometheus.io/probe etc. aren't built in) means everyone's config is probably going to be unique and not portable. For example, we found the current config example to be insufficient, since blackbox-exporter needs information about whether its endpoint is HTTP or HTTPS.
Kubernetes' template system, combined with variable expansion, seems like it would be a better model for what you're currently trying to do with service discovery.
Also: I'm setting this up right now, but it seems there's no exporter for the Kubernetes API proper, just Kubelet?
Is it even right to try to compare this with New Relic? Prometheus looks very interesting and I'm trying to figure out if this is something useful to us. Thanks!
Not quite, though there is some overlap. Someone else asked me to compare Prometheus to other tools in the APM (application performance monitoring) space, and I'm going to share the summary I came up with:
The way I would describe Prometheus in relation to those other tools:
- Prometheus is open-source and self-hosted.
- Prometheus is about dimensional numeric time series metrics only (no log-based analysis, no per-request tracing, etc.).
- Prometheus has a strong focus on systems and service monitoring, not so much on business metrics.
- Prometheus is more of a Swiss army knife of monitoring rather than a ready-to-drop-in package that starts monitoring everything automatically.
- Prometheus is very much about whitebox monitoring and manually defining any metrics that could be useful for you (although we support blackbox exporting and bridging metrics from existing systems as well).
- We don't do machine-learning-style anomaly detection, but we do alerting based on manually defined rules.
- For a purely metrics-based solution, the insight we deliver is one of the best in the field (via the dimensional data model and the query language to go with it).
- Many open-source projects are starting to expose native Prometheus metrics (like k8s, etcd, ...), which gives Prometheus an advantage when being used together with those.
Thanks! I always found NR Servers to be lacking, and frankly, I wanted a more service-oriented monitoring system that tells me how certain services are performing over time. I.e. our API performance, jobs..
What is the typical user/customer of this system ? I've found it very hard to get customers to spend time configuring a motoring system, let alone writing any scripts to gather data.
I'm not sure there is a typical user. We've got everything from single-sysadmin small companies to Fortune 500s to companies looking to integrate it into their products.
Could you elaborate on what seems to be your focus on logging only. Would Prometheus be relevant, for say, high frequency financial markets data? I note that Prometheus "has knowledge about what the world should look like" and "actively tries to find faults" [1]. Isn't this something which is applicable to other fields than simply monitoring? I spend a lot of my time watching and managing "bad data" coming through in finance....
Our focus is not logging only, it's the opposite: we don't support storing logs of individual events. What Prometheus does is store dimensional numerical time series. See the data model: https://prometheus.io/docs/concepts/data_model/
So it's a question of whether you can squeeze your data into that model and whether you need per-event details or whether aggregated time series are ok.
Granted, you don't need to touch this confirmation very often, but anyone who's going to operate the cluster will need to understand it thoroughly.
The fact that it's ad-hoc (prometheus.io/probe etc. aren't built in) means everyone's config is probably going to be unique and not portable. For example, we found the current config example to be insufficient, since blackbox-exporter needs information about whether its endpoint is HTTP or HTTPS.
Kubernetes' template system, combined with variable expansion, seems like it would be a better model for what you're currently trying to do with service discovery.
Also: I'm setting this up right now, but it seems there's no exporter for the Kubernetes API proper, just Kubelet?