[williamcotton]

It’s more like this:

We can take fMRI scans when people are looking at images and generate blurry blobs that do indeed resemble the images spatially.

We can predict a text label of the image the person is looking at using another technique.

If you use SD just on the text labels and you generate an image, you get the semantic content, but not the special content.

If you combine the image and the text label and run it through an LDM then you get pictures that more closely match both the semantic and spatial characteristics of the images shown to the person.

[sillysaurusx]

That’s my understanding as well. It all depends whether their technique really can do this. If it can, it’s solid work imo. If it can’t (better than random chance), then it’s bunk.

There’s not much way to know other than to try it and see. But that’s true of almost every paper in ML. Some of them suck, some of them are great, but they all contribute something in their own way. Even “rat brain flies plane” paper (as much as I despise it) showed that you can change the values of mice neurons in a lab setting.