[trun]

Great article. I've built a number of systems very similar to this and have found Redis to be a fantastic platform, both in terms of reliability and flexibiliy. I'll share a few tips that I think complement your approach.

- When you want to compute metrics for multiple intervals (hour / day / month / etc) Redis' MULTI / EXEC constructs make transactional updates to multiple keys a snap. Additionally batching (which is supported by most Redis clients) can dramatically improve performance.

- You can use Redis sets for computing uniques in realtime. You can also use set operations like SUNION to compute uniques across multiple time periods relatively quickly. For example, SUNION 24 hour intervals to get the total uniques for the day. You just have to be careful that large numbers of uniques eat up your available memory very quickly. EXPIREAT helps ensure things get cleaned up automatically.

- Using a Redis list as an event queue is a great way to further ensure atomicity. Use RPOPLPUSH to move events to a 'uncommitted' queue while processing a batch of events. If you have to rollback, just pop them back on to the original list.

[hdeshev]

And that's why I love Hacker News! Thanks for the tips, trun!

I'll make sure I use batching first, and look into the unions technique after that.