[jcattle]

What I can not wrap my head around: How are these models trained?

What training mechanism or model architecture provides the glue to go from human text to images?

Don't you need to have millions of really descriptively labelled images?

[vonneumannstan]

Short answer yes.

Slightly longer answer for older text to image models you teach them how to encode images and text into the same latent space. Then you simply do a conversion, take a text input, put it into latent space and then extract the image that latent space represents.

[nucleative]

That's exactly how they do it.

There are ML models that do the reverse and output image to text, which assist quite a lot.

The better the text represents the unique thing in the photo, the better the model understands what that text means.

[mistercheph]

I found this 3b1b guest video on diffusion helpful: https://www.youtube.com/watch?v=iv-5mZ_9CPY