You've made two good points. I would suggest the next step is to synthesize them. As with many things, moderation is important. A strategy that balances the two downsides is probably better.
Odds are that each group has a bit of dead wood so taking some from each group would be best - identifying that is the harder part for those several rungs higher