[lambdaone]

You can see a lot of open mouth/tongue stuff being skipped. Dealing with the tongue and the inside of the mouth is a huge problem with this sort of visual dubbing. Using traditional techniques, you can model teeth and gums as rigid bodies, and faces as rubber sheets (to first approximation), but tongues, for which you typically have no visual reference in any given shot, are much more difficult to model, and continuously, subtly, on the move. "AI" is the general answer to this problem nowadays, but even ML-based systems struggle to deal with the tongue issue while trying to reconcile visual appearance with animation fidelity.

[basch]

But the actors re-recorded their dialog in English in the studio. Why wouldn’t you video record their mouths? In fact, the old punch the mouth out and pop the new one in behind the whole can be surprisingly effective sometimes. I would think having an actual visual reference and doing more of a deepfake on the lips should provide excellent results, vs synthetic lip motion to match the audio only.