GMOs are the genetic equivalent to plastic in the ocean. Once it's in the wild it's damn hard to reverse the damage. Weed on the other hand you can just not consume.
GMOs are already produced on a massive scale; any irreversible damage is already done. Further, this is true for any domesticated organism, not GMOs in particular. Besides, the argument I referenced was about health risks, not ecological damage.