[tomgallard]

I have to say, I'm in two minds about this.

On one side- yes it's supercool to have a dashboard with maps and metrics and everything like that. And I can see it could be really useful too when something's going wrong.

On the other hand, I'd be wary of building anything like this until I had a good number of customers, paying me money for a service. Is the product so complete (I would ask myself) and my users so happy, that we have no more important task than building a dashboard to look at in the office. Don't some simple email alerts cover most of the important bases (server down, site down etc) for now.

Your startup needs customers (or failing that, users). Your users don't care if you've got a dashboard, they care that your product does what it says.

[Swizec]

Alert emails can do a lot, but not everything.

Sometimes you need a way to look at the system and say "Hmm, everything works, but those two queues shouldn't have 20k items waiting in them ..."

[fennecfoxen]

Or, even better, "Things aren't working. Why not? Oh, snap, these queues have like 20k items in them. Maybe something's wrong with their consumer."

handy, that one.

[nextmat]

My sense is that people continue to think that monitoring is hard mostly because it used to be. There are great open source tools and services available now to make both collecting the data and watching it pretty simple to set up.

Yes you need some customers, otherwise you have no data to look at. However, as soon as you have _any_ customers if you aren't looking at real data to decide what is important to do next you are essentially just guessing.

For a long time that's what we did because tracking what was relevant was just too hard. Now what I see in a lot of startups that are leading the pack is that they are monitoring everything whether they think they'll need it or not - then when they have a critical question the data is already there.

[sardonicbryan]

Dashboards don't have to be expensive to build. My company uses Graphite now, but before we were at that level, I just had engineering give me credentials for our DBs, and used ODBC + Excel.

Next thing you know, you have a dashboard tracking: - Real time revenue/revenue projections - Real time payments funnel metrics (impressions on payments page, conversions, average transaction size) - Real time tutorial completion rate by hour (aka application health monitoring) - Other KPIs tracked by the day (revenue, install, etc.)

In fact, there was a period of time when my dashboard was the most effective health monitoring in the company, and detected an issue before engineering or ops.

[andraz]

I can't agree more. First you need a product that works at least for some users, then when you are expanding the user base it's time for dashboards.

Dashboard answers the question: Is my product right now working for my users?

Eventually it can also show trends in terms of resource usage, strange peaks, etc. But the primary function of dashboard is answering the question: "are my users getting the service?"

Additional note: dashboard works wonders for getting us - engineers - understand that there are real users out there right now and thus making us want to build even better product.

[viscanti]

This seems pedantic. Of course the product needs to work for some users before you can measure anything about it. Otherwise your dashboard is filled with zeros, which should be obvious because the product doesn't work for anyone (and isn't a product).

Once you've gotten to the MVP stage, you should be tracking the key metrics, and those should be easily viewable in some sort of dashboard. Tracking up-time (or if the service is working) is a small part of what should be measured. When you're just starting off, you need to quickly figure out if your market hypothesis is correct. You can't do that without data, and a dashboard is both easy to build and helps make more informed decisions for everyone involved.

No one is arguing that you should build a dashboard before you have a MVP. Once that happens, you need to be measuring things.

[njx]

Dashboards are not just for "Is my product working for my users?"

They have variety of purposes. For e.g our webapp is an open book, we display live dashboard (https://my.infocaptor.com) of our very app itself because it is meant to be a dashboard platform

The main purpose of dashboards is to keep a birds eye view and take action when something is off.

Don't track metrics where you don't have time or resources to take action against.

[lifeisstillgood]

maybe -

I have taken the time to build a graphite/stasd setup. It is over engineered for now, but it grows and is really easy to add in new metrics. (there is even a fab file to install on ubuntu buried in github/lifeisstillgood/frozone)

but, you are right - a dashboard that measures how the site is going is useful. One that measures cashflow, email conversion rates and more is more useful. Linking those into graphite is not a good idea. A more traditional data collection is needed.

[pcopley]

As someone who has never used Graphite OR statsd, can you explain why you wouldn't want to do this?

[lifeisstillgood]

Graphite/carbon/whisper is a rrd like tool - you give it a number of ticks labelled with a certain metric (facebook.photo.upload) and it counts the incoming ones over a fixed period, averages them, stores the average and goes on and on. Then it draws you a graph over those periods

this is great for "what is normal" and "something has changed over past 5 mins, 5 days, 5 weeks"

some things you want on your dashboard are absolutes that don't vary well with time (cash on hand/ assets vs liabilities)

also the whisper database is awkward to query if you are not aggregating - so you cannot store which customers actually responded to which email campaign - that needs a real RDBMS

So it's really good for watching stuff over time - like you used to use tail -f for. But other stuff needs to be captured robustly and then maybe time graphed

tl dr

Some things are trends - you want to know easily and quickly which ones are going up, and when the trend goes haywire. Use graphite et al. other things are "action this day" - trend or not you need the email addresses of all users on the blue campaign.

Using UDP you lose a not insignificant number as well

[pcopley]

Excellent explanation, thank you.

[Akram]

Cant agree more. I almost always have email triggers built into the system for ever critical error or failure of the system. When we are being lean on the core product itself building less features... build it when you need it approach ... having a dashboard is kind of luxury.

[Kiro]

You're too serious. My users might not care but I do.

[taligent]

I have to respectively disagree.

Metrics are also about the state of your business. What is your growth rate ? What is your churn ? Are people only clicking on certain features ? Where should you improve performance ? These sort of questions are essential for startups especially in the early stages where you need to be careful where to allocate precious resources.

Yes you can sort of gauge that with Google Analytics but only to a certain granularity. Graphite is incredibly flexible.

[ArbitraryLimits]

I think your parent post is saying that metrics are for control and not planning. Once you've identified what your business depends on, and have identified good metrics that (at best) are proxies for those things, a dashboard is a great idea. If on the other hand you don't actually understand your business model, your first goal is to start to understand it, and fixating on a particular set of metrics will only make that harder IMO.