And the solar system is anyway full of flying gravel. The environmental impact is similar if you went to an open mine pit and hammered at a small rock for a while.
I think the concern is that near earth orbit one day will be impassable without some garbage collection.
The acceleration of objects in near earth orbit is vastly greater than the particles from a mine. Space ships collaring with super fast particles in the orbit could spell disaster for the spacecraft.
Hopefully we find ways to make the rock impact negligible if we ever get to the stage of unable to leave orbit due to collisions with space particles (Not saying subatomic particles I just mean debris)
I think you’re both missing marcosdumay‘s point. If you just want to say go to Mars then you can very quickly got far from earth and easily avoid space junk. The IIS is huge and most days has no problem with space debris. It’s only when you start talking about spending months and years in earth orbit that Kessler syndrome becomes a real concern.
Put another way, if you had 1,000 tons of mass to spread around earth orbit it would be difficult to effectively block rockets from leaving. Try to do the same thing with random orbits and it’s even less of a concern.