[currymj]

Generative is a technical term, meaning that a system models a full joint probability distribution.

For example, a classifier is a generative model if it models p(example, label) -- which is sufficient to also calculate p(label | example) if you want -- rather than just modeling p(label | example) alone.

Similar example in translation: a generative translation model would model p(french sentence, english sentence) -- implicitly including a language model of p(french) and p(english) in addition to allowing translation p(english | french) and p(french | english). A non-generative translation model would, for instance, only model p(french | english).

I don't exactly understand what this judge meant by "generative", it's presumably not the technical term.

[jll29]

Your definition of "generative" as a statistical term is correct.

However, and annoyingly so, recently the general public and some experts have been speaking of "generative AI" (or GenAI for short) when they talk about large language models.

This creates the following contradiction:

- large language models are called "generative AI"

- large language models are based on transformers, which are neural networks

- neural networks are discriminative models (not generative ones like Hidden Markov Models)

- discriminative models are the oppositve of generative models, mathematically

So we may say "Generative AI is based on discriminative (not generative) classifiers and regressors". [as I am also a linguist, I regret this usage came into being, but in linguistics you describe how language is used, not how it should be used in a hypothetical world.]

References

- Gen AI (Wikipedia) https://en.wikipedia.org/wiki/Generative_artificial_intellig...

- Discriminative (Conditional) Model (Wikipedia) https://en.wikipedia.org/wiki/Discriminative_model

[echoangle]

Do you have some kind of dictionary where I can find this definition? Because I don’t really understand how that can be the deciding factor of „generative“, and the wiki page for „generative AI“ also seems to use the generic „AI that creates new stuff“ meaning.

By your definition, basically every classifier with 2 inputs would be generative. If I have a classifier for the MNIST dataset and my inputs are the pixels of the image, does that make the classifier generative because the inputs aren’t independent from each other?

[currymj]

if you have an MNIST classifier that just takes in images, and spits out a probability of digits 1-9, that wouldn't necessarily be generative, if it is only capable of modeling P(which digit | all pixels).

But many other types of model would give you a joint distribution P(which digit, all pixels), so would be generative. Even if you only used it for classification.

https://en.wikipedia.org/wiki/Generative_model

I guess these days "generative" must mean "it is used to generate outputs that look like the training data".

But until recently, the meaning had to do with the information in the model, not how it's used.

[zozbot234]

You can derive the latter information (the joint distribution), given the former and a prior over "all pixels"-like data. So, the defining feature of "generative" models is that they feature a prior over their input data?

[Nimitz14]

Generative models model the data, whether that is p(x) or p(x,y) or (x,y,z) etc.

[currymj]

Yes, though maybe not explicitly written down.