"Same difference," as the saying goes. If their claims are true then you can make the model recite "lorem ipsum" or anything else that's long and has nonzero entropy.
It’s not the same. Presumably public domain works are much more frequently shared on the public internet and therefore much more common in the training set