Hacker News new | ask | show | jobs
by BubRoss 2299 days ago
This is an area of research that has been going on for years now, called "natural image matting".

There are dozens of techniques of varying success that have been developed over the course of a decade and a half. My guess is that this is taking some more common implementation like 'closed form matting' and putting it on a server with ffmpeg. To guess the foreground I would use motion vectors as a starting point.

Also note that an alpha channel doesn't get you all the way there. You have to solve the full matting equation to extract both the foreground and alpha. You can see a bright edge around the hair in the example. The result they show still looks pretty good in general though.

1 comments

Pretty sure it's a machine learning model for video segmentation. It doesn't guess the foreground by motion: it guess it with millions of human-annotated masks.

Deep learning is making decades of research obsolete by delivering better results with more generalisation and less time.

Different techniques don't mean it isn't still natural image matting. I was guessing to give people a starting point on what to look at. Does it reference a paper somewhere? Just saying 'deep learning's doesn't really explain much.