Hacker News new | ask | show | jobs
by pogue 642 days ago
Has anyone tried this with some video where they know what the person is saying?

I'd be interested to know how accurate it is, from what angles it will read lips at (front facing, side, etc).

Sounds promising if it works well. Imagine all the historical videos without sound you could try to finally know what was being said.
1 comments
bluGill 642 days ago
Experienced lip readers are lucky to get half of what is said. Better than nothing but not reliable enough for anything and so better to use something else if possible.

'i love you' and 'island view' have the same lip movements is the clasical example.

link
serf 642 days ago
absolutely right.

my mother was a mostly-deaf lip-reader. She needed conversational context in order to keep up 'legibly'; and it created a lot of fun between the two of us when she would come up with an oddball question or comment that had nothing to do with the conversation once-in-awhile when her guesses failed spectacularly.

With context, though, it's a great tool. She and I used to watch crime dramas with the sound off late at night and never miss a beat. It feels if you're trying to transcribe something that has a lot of structural context the success rate is higher than 50%, but I don't know that formally.

It's still a tool I use in conversation. Even with good hearing it's tough to hear people in crowded restaurants or concert venues, lip-reading helps immensely.

link
pogue 640 days ago
What made me think of this was a documentary I saw years ago called "Hitler's Private World" where they used a lip reader and some video enhancements (I don't exactly remember) to be able to read his lips that didn't contain audio, such as those taken at his private villa. I don't know if what they found was ever vetted for accuracy, but it was quite amazing to be able to use lip reading to try and determine what all was being discussed privately.

I found a copy of it on Dailymotion [1] and a brief description [2]. It's well worth a watch! I always wondered why they didn't use these techniques on other recordings of video that have some mystery surrounding them.

[1] https://www.dailymotion.com/video/xlvimo

[2] https://m.imdb.com/title/tt1193023/

link