Nah, I didn't even need scholarships. The financial aid was really good for Stanford undergrad.
My brother went to a state school undergrad, then got grad degrees from Harvard and Georgetown. He's almost done paying off an assload of debt, but it's meant he's had to work as a consultant these last ten years.
I think everybody thought you were talking about "fancy private" elementary/high schools, which is definitely not a middle class thing. Private college/university is a totally different matter.
They have scholarships for private k-12 too. I went to one 6-12 and my family was at no point ever more than lower-middle class. I got what amounted to a full ride almost my entire time there. And even as the family made more still didn't pay much.