[sbierwagen]

(squeezing eyes shut and focusing my psychic powers) This college I've never heard of will turn out to have tiny enrollment, and the entire effect size will be selection bias + random noise.

>Olin College, a small engineering school in Needham, MA, which graduates around 75 students per year, turns out an alumni population where 2.77% of alumni found a successfully venture-backed startup, more than five times the rate of Stanford (0.51%), MIT (0.75%), Harvard (0.28%),

Wow, who would have guessed.

https://fredrikdeboer.com/2017/03/29/why-selection-bias-is-t...

[arcticfox]

Actually, it's not selection bias at all! This result wasn't just drawn out of a 1000-count small college hat, so naive Bayesian priors shouldn't be used here.

As an alum, I specifically went to Olin because I thought it would best help me become a successful entrepreneur. Many people do.

So my priors are that Olin is likely to do this at a higher rate (since Stanford / MIT etc. have entire swathes of the school not interested in entrepreneurship at all); this evidence makes that seem overwhelmingly likely now.

Consider evaluating all the departments of a school and finding out that one of the smallest turns out the highest percentage of high-energy physicists. Selection bias? Nope, just the high-energy physics department.

[sbierwagen]

>Actually, it's not selection bias at all! [...] As an alum, I specifically went to Olin because I thought it would best help me become a successful entrepreneur. Many people do.

From the link:

>In other words, what we might have perceived as a difference in education quality was really the product of systematic differences in how the considered populations were put together. The groups we considered had a hidden non-random distribution. This is selection bias.

As you just stated, entrepreneur-y students self-select into Olin. The fact the school produces entrepreneurs doesn't have anything to do with the teachers, the curriculum, or the chemicals Olin puts in the drinking water. It's the non-random distribution of students.

[pataoAoC]

It depends what the conclusion in question is, whether selection bias is a factor.

Does Olin produce / graduate founders at ~5 times the rate of Stanford? Yes - no bias.

Is the effect size due to self-selection? Likely at least partially.

[valuearb]

If by "partially" you mean "entirely" I agree.

[jwq1]

That is like saying, people who self-select to go to the physics department go there only to be with other physics students are and that it, "doesn't have anything to do with the teachers, [or] the curriculum."

What is more likely, is the school does an excellent job affording students the opportunity found successful, venture-backed companies. Similar to the way the Physics professors and curriculum afford physicists the ability to be good at physics.

The fact the article highlights this opportunity for other entrepreneurially minded people, who may want to attend in the future, is exactly what the data in the article is supposed to do.

[jacobolus]

Stanford also does a good job of helping interested students with those things. It’s just larger and more diverse, and also includes students who want to be judges, literary critics, historians, medical doctors, mathematicians, journalists, school teachers, etc.

If you looked only at the subset of Stanford students with similar interests and backgrounds to Olin’s student population, you’d probably end up with a similar distribution of outcomes.

[ouid]

no, it's called respecting the null hypothesis when making outrageous claims.

[pataoAoC]

I don't agree, I think it's more likely than not that a school specifically designed to foster a particular type of thinking is at least somewhat more effective at it than others. Just like Stanford is likely more effective at developing elite researchers.

I don't think there's nearly enough information to determine that confidently, but we're simply talking about beliefs at this point.

[btilly]

Doubling down on this point, there are undoubtably people who went to Olin not planning to be a founder, who decided to be because they knew someone else who was a good role-model for them.

This is not dissimilar from the old boy network that makes Washington and Lee the single university which most increases your median earning potential. (See https://en.wikipedia.org/wiki/The_Economist_Magazine%27s_Lis... for where I got that from.)

[ucaetano]

> As you just stated, entrepreneur-y students self-select into Olin.

Well, the best students self-select into Berkeley, Harvard, Stanford, etc.

[Tuxer]

"it's not selection bias at all"

"I specifically went to Olin because I thought it would best help me become a successful entrepreneur"

choose one

[pataoAoC]

It depends which conclusion is being questioned.

arcticfox is probably referring to the title conclusion "Olin College Produces Founders at Five Time the Rate of Stanford" not being the result of selection bias during the analysis phase (as implied for example in this comment https://news.ycombinator.com/item?id=15184748).

The conclusion "Olin College is more effective at turning people into founders than Stanford" is indeed a far more speculative conclusion due to self-selection.

[Spooky23]

So would a conclusion that "most sports fans who watch TV watch ESPN" be selection bias?

[dws]

The FIRST Robotics Team I helped out with has sent three students to Olin so far, with more interested. So not only are these students who self-select, but Olin is getting students who've already had an aggressive, entrepreneurial Engineering project under the belts. Our team has sent students to lots of other Engineering schools, but my sense is that Olin is particularly dense with motivated students.

[epalmer]

The incoming class of 2021 http://www.olin.edu/blog/olin-admission/post/welcome-class-2...

[dws]

"And you love robotics! Which makes so much sense- we are Olin after all!- over half of you participated in competitive robotics in high school."

[ComradeTaco]

It is absolutely selection bias. Compare averages by major for each school, not the total population for each school. To compare a college that produces almost exclusively engineering majors to a university that offers everything from constitutional law to communications is absurd.

[jwq1]

To say, “it is absolutely selection bias,” infers the professors and curriculum of the school had no effect on the ability of students to become successful, venture-backed entrepreneurs. Which you cannot conclude given the data.

What you have experienced is confirmation-bias. Whereby, you have interpreted evidence to reaffirm your own beliefs. In this case, your belief is, “Olin College cannot be better than MIT, Harvard, Stanford, Yale, and the like because those schools are the best.”

When you allow yourself to succumb to confirmation bias, you close yourself off to the possibility there is a school which prepares students for an entrepreneurial life. Please do not close others off to the same possibility.

[ComradeTaco]

I never wrote that MIT, Harvard, Stanford and Yale are better than Olin. What I did say was that you should compare apples to apples, compare average start up rate for majors at Olin to the same majors at MIT, Yale, Stanford and Harvard.

Right now most startups are tech heavy. If you have a school with exclusively engineering and technology majors, you're much more likely to have a higher startup rate then a full university.

[pessimizer]

To say "it is absolutely selection bias," you have to include Harvard poetry majors in the comparison, which they did. Why so condescending?

[cjbprime]

Sorry, I guess I'm being slow -- I get that the sample size is too small to confirm the effect, but I'm not seeing where the selection bias is.

Is it that the type of people who would be attracted to founding a startup are more likely to apply/be accepted to Olin than Stanford? That seems to confirm the original article's thesis rather than invalidate it, in my eyes, so I don't expect that's what you're thinking..

[kolbe]

It's possible that Olin exclusively having engineering degrees produces a bias. Stanford has a significant population who study subjects that do not aid in pursuing a career in entrepreneurship.

[sbierwagen]

If you assume that, out of all the students who applied to Olin, only 70 were entrepreneur-y, then a class size of 75 would capture all of them, and Olin would have an impressive number of successful entrepreneurs among its alumni.

But it doesn't scale. If you made Olin accept ten times as many students, then its numbers would regress to the mean.

[rjtobin]

One source of bias here: there are lots of small schools, chances are high that one of them will exhibit a large number of founders due to noise, and this will be the one picked up by the report cited.

[koolba]

With enough tiny schools you can find outliers for anything.

Give me the stats and I'll show you why a seemingly random school is the epicenter of modern opioid addiction.

[btilly]

There aren't enough tiny schools to explain this result.

The p-value against this result if this school was as good as Stanford by pure chance is 1/10k. There are only 3k 4 year colleges in the USA. The vast majority of which are significantly worse than Stanford.

It is therefore extremely unlikely that pure chance alone would explain this data point.

[pbiggar]

I asked my cofounder about this (she's mentioned in the article). She said "This occurred to us. We're a school of engineers - we talk about sample sizes a lot"?

Here's her thoughts on why it's not just sample size: https://twitter.com/ellenchisa/status/905448602142572545

[valuearb]

None of her arguments make any sense. Most apply to any school.

[pbiggar]

Would you care to provide evidence for your aggressively-worded comment?

[bllguo]

Hypothesis 3 - that alumni or faculty are supportive - applies to any number of schools. Need more to show this is a distinguishing factor.

Hypothesis 2 - the school encourages growth and "spiral learning" - is vague. Going to need more evidence to prove this is a unique characteristic of Olin.

Hypothesis 1 - the school attracts students with a certain risk profile - is precisely the parent's point about selection bias. There is a healthy discussion of this above.

[BeetleB]

Your requests for info are fair, but for 2 and 3, it's not hard to find hints of evidence with a mere Google search.

I did not go to Olin, but knew about them from when they were fairly new. Their very existence from the first day has been to produce entrepreneurial students, and their curriculum has always reflected that. IIRC, they don't separate courses based on "engineering" and "entrepreneurship". They aim for every course they teach to be a mix of both. Very few pure "theory" courses. Taking a course in communications? You will learn both the theory (Shannon, etc) and you better build something significant by the end of the semester.

Honestly, from the comments here, I'm surprised at the level of doubt. I thought more people here would be familiar with Olin.

(None of this is to say that they are or are not effective in 2 & 3, but it is a foundational principle of theirs, and I imagine a metric they track - as opposed to typical top schools that claim 3 but are pretty bad at it).

[blfr]

Tiny enrolment is a good point but selection bias is not. You could easily argue that virtually all of college effects are selection effects.

[leblancfg]

I've regularly heard good things of Olin College in the past years, especially when it comes to teaching CS early to all engineering students. Was not surprised to read this headline, but have never met any grads personally.

[ISL]

2/75 = 0.027

[mathattack]

That might be extreme. We'd expect the largest density to come from a small school, but Olin also as the benefit of being 100% engineers with much less debt than comparable schools. That contributes a lot too.

If it were purely size plus noise there would be a few other weird schools on the list.

The bigger question isn't funding, it's how they'll do. No unicorns yet, let alone massive exits. Win the market share for that, and I'll really be impressed.

[jostmey]

And maybe that's how you end up with the harvards and yales. The initial reputation is luck and the schools coast on it. Smart talented people attend but maybe they were going to succeed anyway

[wellok]

Well, it's easy to dismiss. Do you have anything substantive to add to the discussion?