Hacker News new | ask | show | jobs
by thayne 534 days ago
Ok, there are definitely cases where it would be more expensive, like using it for user login data.

But for metrics, like you would use for prometheus:

- Data is usually write-only. There isn't usually any reason to modify the metrics after you have recorded them.

- The bulk of your data isn't going to be used very often. It will probably be processed by monitors/alerts and maybe the most recent data will be shown in dashboard (and that data could be cached on disk or in memory). But most if it is just going to sit there until you need to look at it for an ad-hoc query, and you should probably have an index to reduce how much data you need to read for those.

- This metrics data is very amenable to batching. You do probably want to make recent data available from memory or disk for alerts, dashboards, queries, etc. But for longer term storage it is very reasonable to use chunks of at least several megabytes. If your metrics volume is low enough that you have to use tiny objects, then you probably aren't storing enough to be worried about the cost anyway.