Yes. We do the exact same with disabled students. Yet no one talks about them. I suppose because it is because "looking out for the poor" makes people feel good.
Aren't public schools supposed to have special education resources? I don't know about other schools but the schools I grew up in had disabled students. They just often weren't in the same classes as the higher-achieving kids. But that's no different from students being in different accelerated, AP, or IB classes.