This is the exception, not the rule. And if a cat that's escaped and spends a couple days outside is going to destroy a wildlife ecosystem, then that ecosystem was probably already on the verge of collapse anyway.
I think we have much lower hanging (and more impactful) fruit than "ban even indoor-only cats because sometimes they escape".