It wouldn't practically matter if there were only 500m of us on the planet. I'm not advocating any particular policy btw, but a lot of pollution problems just go away when you minimize the multiplicative factor.
Certain things are possible with 10,000. Others with 10,000,000. Still others with 10 billion.
Certain things are impossible with only 10,000. Others with 10,000,000. Still other things are impossible with 10 billion.
We want rich, diverse, dynamic and satisfying lives. We do not want a crowded, polluted, impoverished world. 10,000 of us is certainly too few to sustain our civilizational accomplishments. My guess is that the numbers we have now are too many. What is the right-ish number?
It doesn't really matter, is my point. We might need at least 30 billion to fully colonise the Solar System one day, or we might need to become a tiny colony that subsists off heat from the Earth's core hiding from super robots.
Ethically, we would need a pressure great enough to reduce the population for a horrible reason. We would never choose it.
It may be that a policy solution can be arrived at that is non-coercive, and gets around the temporary downsides of an aging population. I'm less pessimistic I guess.
We make policies all the time that affect peoples reproductive decisions. Many of these are, in fact, explicitly about biasing reproductive decisions in one way or another. For example, countries like Poland have introduced policies to increase fertility, monetary incentives to have more children. The purpose of that policy is quite explicit. Other policies discourage people from having more children, including improving access to safe and effective contraception, and either the withdrawal of fertility-increasing tax incentives or enabling people to postpone starting a family (which all by itself can slow or reverse population growth).
Yet, it seems that you only have moral qualms on one side of this coin. But perhaps I judge you too hastily.