Who said anything about race? If you can afford Choate, you can afford $250k/year at Harvard. But the kid from Biloxi, MS gets a "scholarship" so she only pays $25k/yr.
Of course, for 250k*4y I could probably just buy a house in Biloxi... And many of the people the current system removes from the pool come from areas that would be just as economically under-performing as other groups.
Less clear than you might think. Choate isn't as rich as Harvard, but it's still on a tier with, say, Wesleyan (for context, not as a fair comparison), with the finaid apparatus to match. They are absolutely capable of letting you "afford" to attend if they want you there, not quite orthogonal to whether you can afford $250k/year (more than Harvard sticker price anyway) but maybe at a 60-degree angle or so.
Sorry if I have missed your point?