[astrange]

It's certainly not true that models are not copyrightable; databases have copyright protection if creativity was involved in creating them.

That said, I don't think outputs of the model are derivative works of it, any more than the model is a derivative of its training data, so it's not clear to me they can actually enforce what you do with them.

[filleokus]

> It's certainly not true that models are not copyrightable; databases have copyright protection if creativity was involved in creating them.

Are you talking about https://en.wikipedia.org/wiki/Database_right or plain old copyright?

I'm no IP lawyer, but I've always thought that copyright put "requirements" on the artefact (i.e the threshold of originality), not the process.

In my jurisdiction we have database rights, meaning that you get IP protections for the artefact based on the work put into the process. For example a database of distances between adress pairs or something is probably not copyrightable, but can be protected under database rights if enough work was done to compile the data.

EDIT: Saw in another place in thread speaking about the https://en.wikipedia.org/wiki/Sweat_of_the_brow doctrine, relates to Database rights. (Neither of which notably are not applicable in the U.S)

[kube-system]

These models are a product of much more creativity than simply a list of phone numbers in a phone book. I don't see how they wouldn't meet the modicum of creativity required for US copyright protection.

[Hizonner]

The software that creates the model is the product of creativity. The model itself is the product of mechanically applying that software to datasets that are (a) assembled with minimal, if any creativity, and (b) definitely not assembled with any eye to the specific form of the resulting model. The whole point is to get the software to form the model without you having to worry about what the result is going to look like. So you can't turn around and claim that the model is a creative work because of the choice of training data.

The only thing that's really specified about the model itself is its architecture, which is (1) dictated by function, and (2) usually deeply stereotyped.

[kube-system]

> mechanically applying that software to datasets that are (a) assembled with minimal, if any creativity, and (b) definitely not assembled with any eye to the specific form of the resulting model.

Fair enough, but those datasets are also primarily copyrighted material. If the software here merely transforms the input material (which I agree it does), then the output is a derivative work.

[Hizonner]

But the pitch-shifted song is still recognizably a creative work. It has identifiable, humanly comprehensible forms of all the original creative elements that Swift originally put into it (plus I guess a de minimis amount of extra creativity from the choice to pitch shift it).

If I take a string of data from a true hardware RNG, XOR it with a Taylor Swift song, and throw away the original random stream, is the resulting fundamentally random bit string still a derivative work of the song? As with the ML model, you can't recognize the song in it. And as with at least some training examples in the inputs of most ML models, you can't recover the song from it either.

It feels like the test for whether X is derivative for copyright purposes should include some kind of attention to whether X is a creative work at all. Maybe not, but then what test do you use?

I do recognize the possibility that the models might not themselves be eligible for copyright as independent works, yet still infringe copyright in the training inputs. It seems messy, but not impossible.

... and as I said elsewhere, it's also messy that while you generally can't recover every training input from the model, you can usually recover something very close to some of the training inputs.

[astrange]

> If I take a string of data from a true hardware RNG, XOR it with a Taylor Swift song, and throw away the original random stream, is the resulting fundamentally random bit string still a derivative work of the song? As with the ML model, you can't recognize the song in it. And as with at least some training examples in the inputs of most ML models, you can't recover the song from it either.

It's not a copy of it, and when you distribute it you're not distributing the original. So it's not a derivative for copyright purposes.

It can still be a derivative for other legal purposes. Judges don't appreciate it when you do funny math tricks like that and will see through them.

> It feels like the test for whether X is derivative for copyright purposes should include some kind of attention to whether X is a creative work at all. Maybe not, but then what test do you use?

Yes, that's how US copyright law works. (well sort of…)

Being a transformative work of something makes it less of a copy of it, the more transformed it is, since it falls under fair use exemptions or is clearly a different category of thing.

If a model was a derivative of its training data, then Google snippets/thumbnails would be derivatives of its search results and would be illegal too. Unless you wrote a new law to specifically allow them.

In other countries (Germany, Japan) fair use is weaker, but model training has laws specifically making it legal in certain circumstances, and presumably so do Google snippets.

[astrange]

There's a lot of effort involved in the training runs too, and you might be able to get away with saying the ops engineers made creative choices too (of which checkpoints were good etc.)

Since it costs millions to produce one of these models, it's not just taking the software and running it to compile them.