[andybak]

> You can't reproduce watermarks that aren't in the training set.

You most certainly can reproduce watermarks that aren't in the training set. It can synthesize novel watermarks the same way it can synthesize novel images.

The vast majority of output is non-infringing.

> It's outside of the rights granted by the copyright holders in most cases (MIT and CC are probably the few licenses which allows this, and even CC/NC wouldn't allow it and MIT falls afoul of the attribution clause).

Except

a) This assumes that training isn't covered under fair use

b) attribution is only required for output that infringes - not the model itself.

Most of your arguments seem to apply to the output not to the model itself or the act of training it.