[dozzie]

Discovery mechanism like this may be nice in some situations, but it doesn't change the underlying architectural problem: you still need to go through central configuration of monitoring system to get something watched.

This architecture results in much longer feedback loop, [object to be monitored -> discovery -> config -> collection engine -> probe -> object's state -> storage] vs. [object to be monitored -> probe -> object's state -> storage]. And you have to move requests/network packets back and forth just to collect the usage for each CPU on a machine separately (or NICs usage, or filesystem usage, or VM/LXC guests, or Apache/nginx vhosts, or...), because the knowledge what objects are there to monitor is on the monitored host's side, not on the monitoring system's side.

And you're limited to whatever resources this predefined discovery mechanism supports. You can't easily write your own probe that discovers things on its own.

And on top of that, discovery mechanism doesn't allow temporary, ad-hoc defined things to be monitored. You can't imagine how often I set a screen with a shell loop to collect a metric or watch state for this one particular process I was debugging:

  while sleep 1; do foo; done | tee /tmp/foo.log

Why not chart that from a monitoring system? Why not display its state on a dashboard? Why not alert in regular way?