I think you're overestimating the fraction of the wall street population that is on the algorithmic or technical side of the game compared to the traditional side that simply provides no value to their clients.
The Yale crowd owns private equity and investment banking. Why? Because private equity and investment banking involve "literally talking to rich old white people" for a living.
Hence they hire the sons of the friends of said old white relics to powwow with them.
However, you can be absolutely sure that the quants pull in the big bucks.