Objectively we’ve only become smarter (IQs have increased), stronger (fitness records continue to be broken), etc. since infant mortality shrank. If you believe otherwise please provide evidence.
Is there a reason you expect IQ to be unbounded? Feels like there are physical constraints, at which point you're sampling from a truncated normal distribution (or similar).
This line of reasoning leads to “some people are weak and unworthy of life because they are impure.” I don’t think rolling back our work to end infant mortality to prevent weak people from living is the right answer.
It doesn't "lead to". Supporting a wrong conclusion even with a valid argument is as old as time. Fans of eugenics don't need an excuse to go down that path.
Our natural and artificial environments shape us for better or worse. We're bigger and more intelligent as a direct outcome of better nutrition. But we also have far more people with deadly allergies because we're so good at managing them now.