[gbrayut]

Is there any way to setup a sharding solution for Prometheus? Like instances per AZ or that only scrape a certain set of nodes? On our larger clusters we had issues even with the optimized Prometheus instance in kubecost, and it's starting to look like metrics collection/processing may be a big driving force on our cluster scaling designs.

[AjayTripathy]

There's definitely ways to do this by configuring your scrape configs to ignore sets of nodes. Curious though when you last tried kubecost out? We've built out some caching mechanisms in the product over the last month or so that should dramatically reduce load / memory consumption on prometheus. If you reach out on our slack kubecost.slack.com we can discuss more about expected Prometheus resource consumption.