Yes, it would make things worse, especially if it comes right after. You wouldn't think "well chemo was already terrible so smoking probably won't make it worse".
You may be right, but it is not obvious (in the case of the sea floor), especially because industrial fishing is so devastating. "Worse" is not enough information, for example if it is "slightly worse" at a point were it's under the variability of the destructing action of fishing. The benefit for the energy transition is to take into account too.
It's worth investigating, in any case. I'm not advocating for taking action before getting the results of such study, though! It is important to be careful.
The only crushing argument that I would admit at this point is: "industrial fishing should not exist in the first place".