[btilly]

The following passage addresses your concern.

We analyzed the results of our first 500 reviews and calculated the correlations between high marks in categories like team, problem area and business economics with the number of requested intros from our reviewing VCs.

In other words the individual factors were graded, and the VC chose whether or not to meet. They are correlating the ranking of the components with the likelihood of asking for a meeting. They are not relying on the VC explanation of why they asked to meet.

I am curious who graded the proposals and how consistent that was. It would also be nice to see what good vs bad looks like in each category.

[brlewis]

Yes, that's an important distinction. At the same time, if VCs graded individual factors, those grades could themselves be rationalizations. For example, they want to meet a founder, and give high marks to the categories they think should be important, not the factor that actually influenced them.

[ceonyc]

They actually weren't "grading" per say. They were answering questions like "Have you seen anything else like this?" and rating teams with multiple choice answers like "This is one of the best teams I've seen." So, they kind of know how a company did, but it's not like they were looking at their own scores at the end.