[davidb_]

Combining this with Motion Amplification/Video Magnification [1] could result in some very interesting visuals and applications for factory equipment.

[1] https://www.youtube.com/watch?v=rEoc0YoALt0 Explainer Youtube video about Motion Amplification

[loxias]

Interesting. I'm casually familiar with Video Amplification (the approach at SIGGRAPH a decade ago IIRC), but have never implemented it myself. A really cool result, using the changes in the phase of the basis vectors over time to infer motion, without having to do dense optic flow.

I'm curious how you would combine acoustic localization in 3 space with motion amplification. I unreservedly agree that they are both "super cool", but don't see how they tie together to make something greater than the sum of their parts.

The only thing I thought of is, if two data channels (video, audio) are registered accurately enough, one could maybe combine the spatially limited frequency information from both channels for higher accuracy?

For example: voxel 10,10,10 is determined (by the audio system) to have a high amount of coherent sound with a fundamental frequency of 2khz. Can that 2khz + 10,10,10 be passed to the video system to do something.... cool? useful? If we know that sound of a certain spectral profile is coming from a specific region, is it useful to amplify (or deaden) video motion with a same frequency?

[elfchief]

I don't suppose you have any idea if there are publicly available motion amplification tools, yet?

[noodlesUK]

A starting point for the MIT research in question can be found here https://people.csail.mit.edu/mrub/vidmag/

[johndough]

The authors of the predecessor method released some of their code:

https://people.csail.mit.edu/mrub/vidmag/#code

[epmaybe]

Motion and color amplification from wu et Al are underused in my opinion. Maybe because under patent?

[johndough]

Patents will expire starting from 2035 up to 2040 depending on the method used.

[5faulker]

Thus another surveillance tool is born.

[fragmede]

It has been rumored that the US military has heartbeat sensors (aka real-life minimap) for decades now, would this really be a new one?

[jcims]

You can pick up heartbeat and breathing rate (if they are relatively still) with simple cw radar, i did it with a $10 HB100 module and audacity.

I bet if you really worked on tuning and filtering you might even be able to pick up the vibration of a persons throat to hear what they say.

[runj__]

Then put a gun on it and you have something even worse!

[Wistar]

That is a great video.