I think he meant more that natural selection is essentially summarized as how many children various groups have. And for some weird reason, natural selection is biasing away from "smart people".
I don’t think it’s too hard to figure out. Natural selection disfavors a strategy that requires significant investment in non-reproductive activity (education, career ladder-climbing) during prime reproductive years.
The selection part works on individuals, what changes from selection is the frequency of alleles in the population. By not having children the alleles you carry will become rarer in the population of the next generation (all things being equal).