Hacker News new | ask | show | jobs
by mukaiji 4981 days ago
Google analytics is great for basic web stuff (getting origin of requests and so forth). KISSMetric is phenomenal as well. If you do email marketing, I think SendGrid might be a good option to measure email engagement metric, on top of not having to maintain an email service.

If ever you feel like doing more data crunching on your own, I recommend getting started with Splunk. The company i work for uses it extensively. Splunk is phenomenal but tricky to use properly (I'll explain). I think the reason it works so well for us is because every action taken by either the server or user-facing devices is carefully logged. As a result, we're able to pull insightful user metrics in highly customizable way that any other service out there just can't get you. Here's the downside of Splunk: since it's so hands-on, there's little initial guidance as to what you should be looking at, and thus the hardest part of Splunk is knowing how to ask the right question. Beyond that, it definitely has its quirks. For example, I wish it was easier to export data to do more in depth data analysis with matlab and the likes (they have a so-so api). I also question the accuracy of their indexing at times (rumors has it around the office that there are log lines that don't get properly fetched...). In any case, if you are serious about analytics (and can afford it in the long run), Splunk is a strong candidate. It will force you to think long and hard about what exactly you are trying to measure.

edit: grammar.

3 comments

I also like to use Splunk for all sort of logs, from nginx to python process logging. Sample of reading nginx log: http://cl.ly/image/1G2D0v3i1b1K/Image%202012-10-27%20at%2010...
We looked at Splunk and liked the architecture, but it doesn't really have customer-focused analytics in its DNA, and it's not open source - which is why we released SnowPlow (https://github.com/snowplow/snowplow)
This is great! The other day I was wondering if anyone had tried to build an open-source Splunk equivalent. I'll look into your guys' stuff. thanks for the ref!
Thanks mukaiji! Any questions you have, just drop me a mail - alex@snowplowanalytics.com
Isn't splunk just used for server performance monitoring or is it a general analytics tool?

At a previous job at a hosting company we used splunk, but we used it for monitoring a couple thousand virtual and physical services. Does it have uses beyond that?

Splunk is good for log processing of all sorts, including event logs. We log app events in a custom format, drain to syslog, and ingest into Splunk. It works great with the exception that some Splunk queries (through the API) can be quite slow for use in online user-facing analytic dashboards, and it's not particularly cheap.
Splunk is a great tool for any sort of logs, and that includes user events and so forth. Basically, if you log it, Splunk will index it, and then you can find ways to search it, correlate it, reverse-analyze it... the whole nine-yards. However, as another user mentioned below, "Log every single action, decision, call, message, visit, and fault in detail. Log it with structure." If you don't do that, Splunk won't be that useful. I can think of a few example at work where missing details in log lines quickly deflates my Splunk enthusiasm. So, log everything and carefully, all the time. Then Splunk becomes your friend.