Hacker News new | ask | show | jobs
by jcims 2273 days ago
Stuff like this can obviously be used to make things like deepfakes 'better'. But i think it might be cool for creating virtual meeting rooms, where you can take a webcam shot of a persons face, normalize the skin tones for lighting, map to a 3d surface, then relight it for the virtual room.

When you can rig the meshes to drive each other you could wear 'masks' of other peoples faces (or critters).
1 comments
st_goliath 2273 days ago
> But i think it might be cool for creating virtual meeting rooms, where you can take a webcam shot of a persons face, normalize the skin tones for lighting, map to a 3d surface, then relight it for the virtual room.

You might be interested in project HeadOn at TU München:

https://www.niessnerlab.org/projects/thies2018headon.html

Justus Thies gave a presentation at our University about a year ago. IIRC they don't use any fancy NN stuff but instead extract the face geometry using a stereo camera and use interpolation to project the movement onto a target mesh. Using stereo goggles for a VR meeting and various other applications were discussed during the presentation, but the main focus was of course on entertaining the audience with fake videos.

On a side note: This is probably the closest to a real-life Max Headroom that we have so far. Not sure if that influenced the name.

link
Frost1x 2273 days ago
These photogrametric, structure from motion, structured light, etc. techniques have been around awhile so I don't think this changes too much though it may make it a bit easier to generate realistic depth maps for other purposes.

At some point you can probably reconstruct large portions of scenes in existing movies and change perspectives, especially if you had good techniques (perhaps AI based) for filling occlusions in the data.

link
vidarh 2273 days ago
One of Vernor Vinge's novels describes 3d models transmitted as part of video conferencing, and re-skinning used to fool an adversary (while depending on causing a noisy, reduced bandwidth connection to make it plausible why the model is imperfect, I seem to remember).
link
jcims 2273 days ago
Oh man, this plus the obs thread earlier plus zoom. The mind boggles haha.
link