[Animats]

The paper, at least as shown here, [1] is vague about which results came from implanted electrodes and which came from functional MRI data. Functional MRI data is showing blood flow. It's like looking at an IC with a thermal imager and trying to figure out what it is doing.

[1] https://archive.is/650Az

[vo2maxer]

Just to clarify, the paper [0] does use both implanted electrodes and fMRI data, but it is actually quite transparent about which data came from which source. The authors worked with two datasets: the B2G dataset, which includes multi-unit activity from implanted Utah arrays in macaques, and the Shen-19 dataset, which uses noninvasive fMRI from human participants.

You’re right that fMRI measures blood flow rather than direct neural activity, and the authors acknowledge that limitation. But the study doesn’t treat it as a direct window into brain function. Instead, it proposes a predictive attention mechanism (PAM) that learns to selectively weigh signals from different brain areas, depending on the task of reconstructing perceived images from those signals.

The “thermal imager” analogy might make sense in a different context, but in this case, the model is explicitly designed to deal with those signal differences and works across both modalities. If you’re curious, the paper is available here:

[0] https://www.biorxiv.org/content/10.1101/2024.06.04.596589v2....

[dogma1138]

If you can extract private keys by measuring how much power a chip consumes I don’t really see a problem with extracting images from fMRI data….

[vo2maxer]

Fair point. Side-channel attacks show how much signal you can pull from noise. But fMRI is a different kind of beast. It’s slow, indirect, and coarse. You’re not measuring neural activity directly, just blood flow changes that lag by a few seconds.

The paper [0] doesn’t pretend otherwise. It trains a model (PAM) to learn which brain regions carry useful info for reconstructing images, and applies this to both fMRI data from humans and intracranial recordings from macaques. The two signal types are handled separately.

If you want an analogy, it’s less like tapping power lines and more like trying to figure out which YouTube video someone is watching by measuring heat on the back of their laptop every few seconds. There’s a pattern in there, but pulling it out takes work.

[0] https://www.biorxiv.org/content/10.1101/2024.06.04.596589v2....

[buildbot]

That could be an interesting project in itself, take a simple 8 but microcontroller, a thermal camera, and some code that does different kinds of operations, see if you can train a classification model at least, or even generate the code running via an image to text llm.

[tharant]

Please stop giving me project ideas. :)

[moffkalast]

Ah yes, yet another attack vector chip manufacturers will have to protect against now.