[pg]

Exits are a stupid test for what they're trying to measure here. By that standard, Airbnb and Dropbox are failures.

Exits are a reasonable test for investments made, say, 10 years ago. But none of the incubators are that old yet. So the right way to judge them is by the valuations of the startups they've funded. Unless the venture business as a whole loses money, that will be a lower bound on the eventual exit numbers.

Then you don't need to measure fuzzy stuff like "VC perceptions" either. Each incubator has a single score: average valuation. The last time we calculated ours (for the Forbes incubator rankings: http://www.forbes.com/sites/tomiogeron/2012/04/30/top-tech-i...) it was $45.2m.

You could still screw up e.g. in the case where a company hasn't raised money for a long time and whose last post-money valuation is 1/10 of what they could raise at now. But you won't screw up as badly as if you just measure acquisitions.

[kylebrown]

How about median valuation?

[pg]

The trouble with median valuation is that even for the most successful incubators it will approach zero, because I doubt any of us are going to have a success rate over 50%.

[seats]

One thing that is interesting about looking at both mean and median is getting a feel for the shape/variance of the returns.

One huge winner could randomly happen in an otherwise low-quality program and that incubator would appear higher on rankings forever after that unjustly since the result is very much not reproducible. In other words measuring by average could mask getting lucky.

[throwaway1979]

"One huge winner could randomly happen ..."

One would think that. The article states that 45% of programs have not had any graduate that raised venture funding. That boggles my mind. But this data helps explain the uneasy feeling I get when I see niche accelerators following a cargo cult mentality.

[waterlesscloud]

Hmm. But isn't that getting at the point they're making here?

[tptacek]

You'd hope not, because that would be a silly point to make. No reasonable observer expects the majority of any portfolio of startups to succeed.

[TimGebhardt]

But VC investing is a hit-driven business. One big hit, like a Dropbox, funds all the rest. Unless you're really good at picking horses, you'll have the same median as everyone else.

[kylebrown]

That's why I liked dan shipper's (rather timely) blog post the other day about choosing to build a sustainable business instead of swinging for a homerun, and striking out.

[tptacek]

If you're an entrepreneur, this is indeed the way you want to be thinking. But these are investors discussing outcomes for different startup creation vehicles. If you're an investor, you need to focus on the skinny tail of the outcome distribution, if only because it's where the liquidity is.

[dxbydt]

While I applaud his motivation, what he's suggesting is an intrinsically unsound way to go about it. A startup = a chemical reaction with extremely high activation energy. In such circumstances, the best thing you can do is get one or more catalysts. YC or any other VC/incubator is just a catalyst to lower your activation energy & let the chemical reaction happen sooner/faster. Without the catalyst, the reaction will most likely not happen, period. Whether you get a catalyst or not, you most definitely don't want inhibitors in your reaction. "Holed up in Philly for $650 a month working 14-hours a day" - that's a bigtime inhibitor right there. Why not spend $6500 in sv and work 8 hours a day - its a lot more sustainable, that $6500 comes out of some vc's pocket in exchange for equity, nobody's burnt out and everybody's happy. Let the capital markets work in your favor. Don't handicap yourself.

[waterlesscloud]

But success for purposes of a median is a low bar. It's half your companies having any valuation at all.

[PeterisP]

The whole point is that a very great fund will still have 60%+ with valuation of 0, and that's completely ok.

[Evbn]

Median is arbitrary. Why not 90th percentiles? Profit pays the bills.

[Daniel_Newby]

What about comparing the 85-95 percentile companies? Is there any point to comparing the companies that have modest success?

[michael_fine]

Isn't measuring by average valuation a little dissonant with how the valuations play out? If startup valuations follow a power law, and most of the money is made from a few successful exits, wouldn't it make more sense to judge incubators by say, their top 10%, while also showing the total number of startups for comparison?

[pg]

If you're measuring them as investors, you want the average, because if (as all these incubators do) they invest roughly the same amount in every startup, then average is money out divided by money in.

[startupfounder]

Even saying it is about the average valuation doesn't take into account the age of the accelerator. This is the real problem I see.

What is the average valuation of each class after the same time period? For example, the first class of YC vs. TS vs. SC vs. etc. after 1, 3 and 5 years?

This would show us the speed of growth of each accelerator compared to other accelerators when they were the same age. Even if YC has a higher average valuation now there might be an accelerator out there that has better valuation growth, but it is hidden because we are comparing baby apples to apples almost ready to harvest (sneaky apples to apples reference).

So yes average, but average comparing apples to apples.

Oooo... time for an apple :)

[pg]

Yeah, actually, come to think of it the right metric might be valuation divided by age in years.

[rudiger]

What about the founders? They don't have the liberty of hedging their bets, so for them it can be a raw deal.

[larrys]

Here is the slideshare on the project.

http://www.slideshare.net/dgiluz/accellerators-in-us-and-eur...

No mention of the 10 VC's that I can find.

[snambi]

If the startup is profitable and self sustain itself, does it count as success or failure?

[pg]

To the algorithm they use in this study, it counts as a failure unless the company is public, which is my point here.

To investors, whether an investment is a success or a failure is indeterminate till the company either goes out of business or returns the capital invested.

[snambi]

when the company is profitable and has valuation, the investors can sell their shares. If the investor makes a profit by selling their shares, will it count still count as failure? It is possible that the investor can lose money, even if the company goes public, right?

[pg]

I don't understand what you're asking. Can you try again?

[snambi]

Lets say an investor puts 3M for 30% of the company. 1-2 years later, X is profitable and valued at 30M. Now, if the investor sells their 30%, they will make 9M. That is 6M in profits.

Is this considered as success? It is not an IPO or big acquisition. But, the investor can make money by selling their shares.

Is this something the investors consider doing?

[arockwell]

In practice it is very difficult to sell shares of a private company.

[raverbashing]

"self sustain" is a complicated concept

It can mean "ramen profitable" or something slightly better (like it can pay the rent and the founders a small salary)

Either way, not good. You don't need a home run, but you need something that can "fly for itself""

Getting there is tough, most startups are like making a rock fly

[brittohalloran]

Assuming it's actually making good money not just breaking even, its true valuation will be high (though it can't be measured using the 'latest round' method)