[Rapzid]

Very interesting.. I'm guessing you can do less than a minute on granularity? That's a lifetime to me.

I'm working on my own little system based on Riak for storage and their new search functionality to index the streams(what I call a unique metric). I have a goal of graphite API compatibility though so it can be a drop-in replacement for me. Worked out a schema for mapping graphite metric names to the stream dimensions, wrote a graphite function parser/lexar, etc. Atlas should definitely be worth a look for some fresh ideas :)

[CrankyFool]

We CAN do <1m, but very rarely do (at least in terms of persisting and showing it). We have a feature called 'Critical Metrics' that is a separate publishing pipeline into Atlas that is shorter, simpler, hardier, and supports 10s granularity, though for a trivial minority of metrics -- our current limit is on the order of around 400K metrics per cluster, IIRC, which means that if you've got, say, 1000 nodes, we're going to limit you to 400 metrics per node that would be flowing through the Critical Metrics pipeline.

(We haven't opensourced much of the pipeline ... yet)