Being the exception doesn't exclude the fact that poor kids are at disadvantage of gaining admissions to Ivies. A student from Phillips Exeter Academy has a much higher chance of being accepted vs a very smart but poor kid from a rural America high school.
As an example of this comment's point, one of the problems where I grew up is that when you told the guidance counselor you wanted to go to MIT she said, basically, don't bother.
She probably said that because of all the people pushing the narrative that elite schools are only for rich kids. Spreading this myth dissuades thousands of qualified students from applying to top schools where they would get a great education for free.